Home | History | Annotate | only in /src/sys/arch/aarch64/include
History log of /src/sys/arch/aarch64/include
RevisionDateAuthorComments
 1.5 30-Nov-2024  christos Create a new header lwp_private.h to contain _lwp_getprivate_fast,
_lwp_gettcb_fast, _lwp_settcb and remove them from mcontext.h, so that:
1. we don't need special hacks to hide them
2. we can include <lwp.h> where needed to get the necessary prototypes
without redefining them locally.
 1.4 10-May-2020  skrll branches: 1.4.26;
Provide a trap.h (currently empty)
 1.3 09-Dec-2018  alnsn branches: 1.3.4;
Install aarch64/sljit_machdep.h.
 1.2 01-Apr-2018  ryo branches: 1.2.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.2 26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file Makefile was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2.2.1 10-Jun-2019  christos Sync with HEAD
 1.3.4.1 13-May-2020  martin Pull up following revision(s) (requested by skrll in ticket #900):

sys/arch/aarch64/include/Makefile: revision 1.4
sys/arch/aarch64/include/trap.h: revision 1.3
distrib/sets/lists/comp/ad.aarch64: revision 1.40

Provide a trap.h (currently empty)

Update for trap.h
 1.4.26.1 02-Aug-2025  perseant Sync with HEAD
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file ansi.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file aout_machdep.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.71 23-Aug-2025  skrll Add a #define for the RES1 bits (just bit 31) in MPIDR_EL1.
 1.70 12-Aug-2025  skrll Add MDCR_EL2 accessors and bit definitions.
 1.69 12-Aug-2025  skrll Add sp_el2 accessors.
 1.68 12-Aug-2025  skrll Remove the XXXNH I had against ESR_EC_LS64.

The FEAT_LS64 instructions are all A64.
 1.67 27-Feb-2025  andvar Fix various typos in comments.
 1.66 03-Jan-2024  andvar branches: 1.66.2;
ddress->address in comment.
 1.65 24-Sep-2023  skrll Add a bunch of system registers and their bit / bit field definitions.
Taken from ryo's nvmm branch with updates from me.
 1.64 06-May-2023  andvar s/Regiser/Register/ and s/regester/register/ in comments.
 1.63 01-Dec-2022  ryo Improve tprof(4)

- Multiple events can now be handled simultaneously.
- Counters should be configured with TPROF_IOC_CONFIGURE_EVENT in advance,
instead of being configured at TPROF_IOC_START.
- The configured counters can be started and stopped repeatedly by
PROF_IOC_START/TPROF_IOC_STOP.
- The value of the performance counter can be obtained at any timing as a 64bit
value with TPROF_IOC_GETCOUNTS.
- Backend common parts are handled in tprof.c as much as possible, and functions
on the tprof_backend side have been reimplemented to be more primitive.
- The reset value of counter overflows for profiling can now be adjusted.
It is calculated by default from the CPU clock (speed of cycle counter) and
TPROF_HZ, but for some events the value may be too large to be sufficient for
profiling. The event counter can be specified as a ratio to the default or as
an absolute value when configuring the event counter.
- Due to overall changes, API and ABI have been changed. TPROF_VERSION and
TPROF_BACKEND_VERSION were updated.
 1.62 01-Dec-2022  ryo PMCR.E should not be disabled from tprof.

PMCR.E controls not only performance event counters but also the cycle
counter operation, and the cycle counter may be used for cpu_counter.
Similarly, the 31st bit in PMINTENCLR and PMCNTENCLR controls the cycle
counter, not performance event counters, and should not be modified.
 1.61 02-May-2022  skrll Only print the appropriate PAR fields for PAR.F={0,1}

Group the fields in the header.
 1.60 05-Jan-2022  ryo fix ID_AA64ISAR0_EL1.ATOMIC field definition
 1.59 26-Oct-2021  ryo fix build with COPTS=-O0
 1.58 23-Oct-2021  skrll Typo in comment
 1.57 19-Jun-2021  jmcneill Do not try to initialize PMU if ID_AA64DFR0_EL1 reports a non-standard
PMU implementation.
 1.56 19-Jun-2021  jmcneill CNTV_CTL_EL0 is a 64-bit register
 1.55 09-Mar-2021  ryo branches: 1.55.4;
fixed mask width of DBGWVR_MASK, and added definition of DBGBVR_MASK
 1.54 30-Sep-2020  ryo branches: 1.54.2;
add some fields of ID_AA64ISAR1_EL1 definition (ARMv8.6)
 1.53 15-Sep-2020  ryo fix typo
 1.52 02-Aug-2020  maxv Add support for Privileged Access Never (ARMv8.1-PAN).

PAN provides the same functionality as SMAP on x86: it forbids kernel
access to userland pages when PSTATE.PAN=1, and allows such accesses when
PSTATE.PAN=0.

We clear SCTLR_SPAN, to guarantee that PAN=1 each time the kernel is
entered. We catch PAN faults and panic right away without further
processing. In copyin, copyout, etc, we temporarily authorize access to
userland pages.

PAN is a very useful exploit mitigation. Reviewed by ryo@, thanks. Tested
on Qemu. Enabled by default.
 1.51 01-Aug-2020  maxv The system registers we modify can have an impact on memory accesses, and
we don't want the compiler to randomly re-order the instructions, so add
barriers. Same as WRMSR on x86.
 1.50 01-Jul-2020  ryo - On some systems with a different cache line size (and DIC,IDC) per CPU, trap "mrs Xt,ctr_el0" instruction
to return the minimum cache line size of the system to userland.
- add CLIDR_EL1 and CTR_EL0 to struct aarch64_sysctl_cpu_id.

On most systems, cache line size is the same for all CPUs, so this mechanism won't be required.
Rather, this is primarily for errata support, which will be committed later.
 1.49 14-Jun-2020  riastradh Add some more id_aa64pfr0_el1 bits.
 1.48 28-May-2020  skrll Add some new CTR_EL0 bits
 1.47 25-May-2020  ryo add ARMv8.1-8.5 definitions of TCR_EL1
 1.46 25-May-2020  ryo cache information can be detected correctly on newer CPUs

- add VPIPT cache type
- adapt to 64-bit CCSIDR (ARMv8.3-CCIDX)
- CCSIDR:[WT,WB,PA,WA] are deprecated
- show number of cache lines when attaching cpu
 1.45 23-May-2020  ryo Not only the kernel thread, but also the userland PAC keys
(APIA,APIB,APDA,APDB,APGA) are now randomly initialized at exec, and switched
when context switch.
userland programs are able to perform pointer authentication on ARMv8.3+PAC cpu.

reviewd by maxv@, thanks.
 1.44 21-May-2020  ryo fix typo
 1.43 13-May-2020  ryo - move aarch64 addressspace macros from pmap.h to cpufunc.h
- rename ptr_strip_pac() to aarch64_strip_pac()
 1.42 11-May-2020  ryo "options ARMV83_PAC" is now supported for gcc as well.

- add "-msign-return-address=all" to CFLAGS for gcc when specified options ARMV83_PAC
- AARCH64REG_{READ,WRITE}_INLINE3 macro can now use the APIAKey registers in both gcc and llvm.
llvm requires asm(".arch armv8.3-a"), whereas gcc requires __attribute__((target("arch=armv8.3-a"))).
- use ".arch armv8.3-a" rather than ".arch armv8.3-a+pac" in *.S for llvm.
 1.41 10-May-2020  riastradh Fix ID_AA64ISAR0_EL1_RNDR field definition for RNDR support.

ARMv8.5 ARM, p. D13-3232
 1.40 12-Apr-2020  maxv Add support for Pointer Authentication (PAC).

We use the "pac-ret" option, to sign the return instruction pointer on
function entry, and authenticate it on function exit. This acts as a
mitigation against ROP.

The authentication uses a per-lwp (secret) I-A key stored in the 128bit
APIAKey register and part of the lwp context. During lwp creation, the
kernel generates a random key, and during context switches, it installs
the key of the target lwp on the CPU.

Userland cannot read the APIAKey register directly. However, it can sign
its pointers with it, because the register is architecturally shared
between userland and the kernel. Although part of the CPU design, it is
a bit of an undesired behavior, because it allows to forge valid kernel
pointers from userland. To avoid that, we don't share the key with
userland, and rather switch it in EL0<->EL1 transitions. This means that
when userland executes, a different key is loaded in APIAKey than the one
the kernel uses. For now the userland key is a fixed 128bit zero value.

The DDB stack unwinder is changed to strip the authentication code from
the pointers in lr.

Two problems are known:

* Currently the idlelwps' keys are not really secret. This is because
the RNG is not yet available when we spawn these lwps. Not overly
important, but would be nice to fix with UEFI RNG.
* The key switching in EL0<->EL1 transitions is not the most optimized
code on the planet. Instead of checking aarch64_pac_enabled, it would
be better to hot-patch the code at boot time, but there currently is
no hot-patch support on aarch64.

Tested on Qemu.
 1.39 30-Mar-2020  jmcneill branches: 1.39.2;
Enable the cycle counter when a CPU hatches and store an estimate of the
frequency in ci_data.cpu_cc_freq.
 1.38 06-Mar-2020  ryo fix missing paren
 1.37 06-Mar-2020  ryo add more definitions for ARMv8.1-ARMv8.4
 1.36 29-Feb-2020  ryo widen bit PAR_EL1.PAR_PA from [47:12] to [51:12] for ARMv8.2 (and later).

PAR_EL1:[51:48] is RES0 in ARMv8.1 and ARMv8.0.
 1.35 31-Jan-2020  maxv BTI definitions.
 1.34 28-Jan-2020  maxv More SCTLR.
 1.33 28-Jan-2020  maxv Fetch ID_AA64MMFR2_EL1. Okayed by Nick the other day.
 1.32 28-Jan-2020  maxv Jazelle and T32EE are not part of ARMv8, fix the bits to their real
meanings. No functional change.
 1.31 28-Jan-2020  maxv More definitions.
 1.30 28-Dec-2019  rjs branches: 1.30.2;
s/Memroy/Memory/ in comment.
 1.29 27-Dec-2019  jmcneill Enable early write acknowledge for device memory mappings.
 1.28 15-Sep-2019  tnn report A72 errata #859971 workaround status during boot
 1.27 11-Sep-2019  skrll Move the TCR and TTBR defines into armreg.h where they below. NFCI.
 1.26 12-Aug-2019  jmcneill Add support for physical timers and sprinkle isb where needed.
 1.25 16-Jun-2019  skrll branches: 1.25.2;
Provide icc_pmr_read
 1.24 20-Mar-2019  ryo - add reg_{s1e0r,s1e0w,s1e1r,s1e1w}_write() macro.
- show the result of AT insn at ddb "machine pte" command.
 1.23 30-Jan-2019  jmcneill add gtmr_cntv_cval_write
 1.22 13-Dec-2018  ryo add support PT_STEP
 1.21 20-Nov-2018  mrg rewrite the CPU identification on arm64:

- publish per-cpu data
- publish a whole bunch of info in struct aarch64_sysctl_cpu_id
instead of various individual nodes (there are 16 total.)
- add MIDR extractor bits
- define ARMv8.2-A id_aa64mmfr2_el1 and id_aa64zfr0_el1 regs,
but avoid using them until we make sure they exist. (these
members are added to aarch64_sysctl_cpu_id to avoid future
compat issues.)

the arm32 and aarch32 version of these need to be adjusted as
well (and aarch32 data published at all.) still trying to
work out how to make the same userland binary running on a
real arm32 or an aarch32 system can work sanely here.

ok ryo@.
 1.20 07-Nov-2018  riastradh When hardware subnormal support is available, disable flush-to-zero.

Similarly, when hardware NaN propagation is available, disable
default-NaN substitution.

This enables IEEE 754 semantics on any hardware that supports it by
default. Programs that want flush-to-zero or default-NaN substitution
can enable them explicitly.

ok ryo@
 1.19 12-Oct-2018  ryo add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.
 1.18 12-Aug-2018  skrll Provide and use cpu_mpidr_aff_read in psci_fdt_bootstrap
 1.17 12-Aug-2018  skrll Whitespace
 1.16 09-Aug-2018  jmcneill Restore ICC_SRE_EL2 registers lost in previous commit
 1.15 08-Aug-2018  jmcneill Add GICv3 system registers
 1.14 05-Aug-2018  skrll More whitespace
 1.13 01-Aug-2018  skrll Some whitespace improvements. NFC.
 1.12 17-Jul-2018  christos - use #define to define constants instead of static const variables so that
gcc can compile the code.
- fix position of inline, and use __inline
 1.11 15-Jul-2018  jmcneill Add more PMC registers
 1.10 14-May-2018  joerg branches: 1.10.2;
Workaround A-008585 errata in GTMR.

Register reads and writes may provide unstable results if the counter
hardware is active at the same time. This results in non-monotonic
counters seen by both the gtmr interrupt and time counter.

The loops are currently applied unconditionally, restricting them to
appropiate FDT markers can be applied later.
 1.9 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.8 20-Mar-2018  ryo separate cputypes.h for CPU_ID_* from armreg.h,
and add some implementor IDs, CortexA55,73,75 IDs.

(preliminary changes for merging aarch64)
 1.7 06-Mar-2018  skrll Sprinkle __volatile on asm instructions
 1.6 06-Mar-2018  skrll Convert decimal to hex to make comparison to arm arm (slightly) easier.
 1.5 06-Mar-2018  skrll Another harmless typo
 1.4 06-Mar-2018  skrll Fix harmless typo
 1.3 20-Dec-2017  skrll branches: 1.3.2;
Trailing whitespace
 1.2 27-Apr-2015  skrll ARM spells the System Control Register SCTLR
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.6;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.6.1 06-Jun-2015  skrll Sync with HEAD
 1.1.4.3 03-Dec-2017  jdolecek update from HEAD
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file armreg.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.3.2.9 26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.3.2.8 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.3.2.7 20-Oct-2018  pgoyette Sync with head
 1.3.2.6 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.3.2.5 28-Jul-2018  pgoyette Sync with HEAD
 1.3.2.4 21-May-2018  pgoyette Sync with HEAD
 1.3.2.3 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.3.2.2 22-Mar-2018  pgoyette Synch with HEAD, resolve conflicts
 1.3.2.1 15-Mar-2018  pgoyette Synch with HEAD
 1.10.2.3 21-Apr-2020  martin Sync with HEAD
 1.10.2.2 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.10.2.1 10-Jun-2019  christos Sync with HEAD
 1.25.2.2 29-Dec-2019  martin Pull up following revision(s) (requested by jmcneill in ticket #586):

sys/arch/arm/nvidia/tegra_pcie.c: revision 1.27
sys/arch/aarch64/aarch64/pmap.c: revision 1.57
sys/arch/aarch64/aarch64/locore.S: revision 1.48
sys/arch/aarch64/include/armreg.h: revision 1.29
sys/arch/aarch64/aarch64/pmap.c: revision 1.58
sys/arch/aarch64/aarch64/locore.S: revision 1.49
sys/arch/arm/acpi/acpipchb.c: revision 1.14
sys/arch/aarch64/aarch64/genassym.cf: revision 1.16
sys/arch/arm/acpi/acpi_machdep.c: revision 1.13
sys/arch/aarch64/include/pmap.h: revision 1.27
sys/arch/aarch64/aarch64/genassym.cf: revision 1.17
sys/arch/aarch64/include/pmap.h: revision 1.28
sys/arch/arm/fdt/pcihost_fdtvar.h: revision 1.3
sys/arch/arm/include/bus_defs.h: revision 1.14
sys/arch/aarch64/aarch64/bus_space.c: revision 1.9
sys/arch/arm/fdt/pcihost_fdt.c: revision 1.12
sys/arch/aarch64/conf/files.aarch64: revision 1.15
sys/arch/aarch64/conf/files.aarch64: revision 1.16
sys/arch/arm/rockchip/rk3399_pcie.c: revision 1.9

Enable early write acknowledge for device memory mappings.

Do not use Early Write Acknowledge for PCIe I/O and config space.
 1.25.2.1 13-Aug-2019  martin Pull up following revision(s) (requested by jmcneill in ticket #54):

sys/arch/aarch64/include/armreg.h: revision 1.26
sys/arch/arm/cortex/gtmr.c: revision 1.41
sys/arch/arm/include/armreg.h: revision 1.128
sys/arch/arm/cortex/gtmr_var.h: revision 1.12

Add support for physical timers and sprinkle isb where needed.
 1.30.2.1 29-Feb-2020  ad Sync with head.
 1.39.2.1 20-Apr-2020  bouyer Sync with HEAD
 1.54.2.1 03-Apr-2021  thorpej Sync with HEAD.
 1.55.4.1 01-Aug-2021  thorpej Sync with HEAD.
 1.66.2.1 02-Aug-2025  perseant Sync with HEAD
 1.19 16-Apr-2023  skrll Rename VM_KERNEL_IO_ADDRESS to VM_KERNEL_IO_BASE to match RISC-V

It's less letters, matches other similar variables and will help with
sharing code between the two architectures.

NFCI.
 1.18 29-Apr-2021  skrll Remove some unnecessary tlb invalidate in pmap_growkernel and ASAN shadow
map. Ensure the shadow map mappings are visible to the TLB walkers.
 1.17 21-Mar-2021  skrll branches: 1.17.2;
Adjust the kernel virtual address space so that KASAN will map the kernel
seperately from managed kernel virtual memory and not map the unused space
between the two.
 1.16 11-Dec-2020  skrll branches: 1.16.2;
s:aarch64/cpufunc.h:arm/cpufunc.h:

a baby step in the grand arm header unification challenge
 1.15 26-Nov-2020  skrll Mark KASAN shadow pages as LX_BLKPAG_ATTR_NORMAL_WB. NFC as this is zero,
but someone might change it one day.
 1.14 10-Nov-2020  skrll AA64 is not MIPS.

Change all KSEG references to directmap
 1.13 20-Sep-2020  skrll branches: 1.13.2;
Use pmap_growkernel(VM_KERNEL_VM_BASE) rather than pmap_virtual_space to
work out what to map initially.

XXX could do better mapping the kernel and modules more accurately
 1.12 19-Sep-2020  skrll Make __md_palloc pmap agnostic (think sys/uvm/pmap)
 1.11 10-Sep-2020  maxv kasan: fix the copyright notices
 1.10 05-Sep-2020  riastradh Round of uvm.h cleanup.

The poorly named uvm.h is generally supposed to be for uvm-internal
users only.

- Narrow it to files that actually need it -- mostly files that need
to query whether curlwp is the pagedaemon, which should maybe be
exposed by an external header.

- Use uvm_extern.h where feasible and uvm_*.h for things not exposed
by it. We should split up uvm_extern.h but this will serve for now
to reduce the uvm.h dependencies.

- Use uvm_stat.h and #ifdef UVMHIST uvm.h for files that use
UVMHIST(ubchist), since ubchist is declared in uvm.h but the
reference evaporates if UVMHIST is not defined, so we reduce header
file dependencies.

- Make uvm_device.h and uvm_swap.h independently includable while
here.

ok chs@
 1.9 01-Aug-2020  maxv Use large pages for the KASAN shadow, same as amd64, discussed with ryo@.
 1.8 16-Jul-2020  skrll pmapboot_enter simplication
- bootpage_alloc in asm becomes pmapboot_pagealloc in C
- PMAPBOOT_ENTER_NOBLOCK is removed as it's not used
- PMAPBOOT_ENTER_NOOVERWRITE is removed as it's now always on
- physpage_allocator argument is removed as it's always
pmapboot_pagealloc
- Support for EARLYCONS without CONSADDR is removed so that the identity
map for CONSADDR is always known.

For the assembly files:
2 files changed, 40 insertions(+), 89 deletions(-)

LGTM ryo
 1.7 23-Jun-2020  maxv Rename __MD_CANONICAL_BASE -> __MD_KERNMEM_BASE for clarity.
 1.6 08-Apr-2019  ryo branches: 1.6.4;
- free empty page tables pages if reach a certain usage.
- need to lock at removing an old pg (_pmap_remove_pv) in _pmap_enter()
 1.5 19-Mar-2019  ryo - free L1-L3 pages that has been emptied by pmap_remove().
- if no memories, pmap_enter will return correctly ENOMEM if PMAP_CANFAIL, or wait until available any memories if !PMAP_CANFAIL.

These changes improves the stability when we use a huge virtual memory spaces with mmap.
 1.4 10-Nov-2018  ryo branches: 1.4.2;
add LX_BLKPAG_SH_IS pte attribute for MP
 1.3 08-Nov-2018  maxv Track the stack with kASan on aarch64. Same principle as on amd64. Illegal
accesses occurring there are now detected.

Originally written by me, but reworked by ryo@, thanks.
 1.2 02-Nov-2018  skrll Provide a kasan_md_unwind

OK maxv
 1.1 01-Nov-2018  maxv Add kASan support for aarch64. Stack tracking needs more investigation
and will come in a separate commit.

Reviewed by ryo@ jmcneill@ skrll@.
 1.4.2.2 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.4.2.1 10-Nov-2018  pgoyette file asan.h was added on branch pgoyette-compat on 2018-11-26 01:52:16 +0000
 1.6.4.2 10-Jun-2019  christos Sync with HEAD
 1.6.4.1 08-Apr-2019  christos file asan.h was added on branch phil-wifi on 2019-06-10 22:05:43 +0000
 1.13.2.2 03-Apr-2021  thorpej Sync with HEAD.
 1.13.2.1 14-Dec-2020  thorpej Sync w/ HEAD.
 1.16.2.1 03-Apr-2021  thorpej Sync with HEAD.
 1.17.2.1 13-May-2021  thorpej Sync with HEAD.
 1.9 02-Aug-2020  maxv Add support for Privileged Access Never (ARMv8.1-PAN).

PAN provides the same functionality as SMAP on x86: it forbids kernel
access to userland pages when PSTATE.PAN=1, and allows such accesses when
PSTATE.PAN=0.

We clear SCTLR_SPAN, to guarantee that PAN=1 each time the kernel is
entered. We catch PAN faults and panic right away without further
processing. In copyin, copyout, etc, we temporarily authorize access to
userland pages.

PAN is a very useful exploit mitigation. Reviewed by ryo@, thanks. Tested
on Qemu. Enabled by default.
 1.8 11-May-2020  ryo "options ARMV83_PAC" is now supported for gcc as well.

- add "-msign-return-address=all" to CFLAGS for gcc when specified options ARMV83_PAC
- AARCH64REG_{READ,WRITE}_INLINE3 macro can now use the APIAKey registers in both gcc and llvm.
llvm requires asm(".arch armv8.3-a"), whereas gcc requires __attribute__((target("arch=armv8.3-a"))).
- use ".arch armv8.3-a" rather than ".arch armv8.3-a+pac" in *.S for llvm.
 1.7 12-Apr-2020  maxv Add support for Pointer Authentication (PAC).

We use the "pac-ret" option, to sign the return instruction pointer on
function entry, and authenticate it on function exit. This acts as a
mitigation against ROP.

The authentication uses a per-lwp (secret) I-A key stored in the 128bit
APIAKey register and part of the lwp context. During lwp creation, the
kernel generates a random key, and during context switches, it installs
the key of the target lwp on the CPU.

Userland cannot read the APIAKey register directly. However, it can sign
its pointers with it, because the register is architecturally shared
between userland and the kernel. Although part of the CPU design, it is
a bit of an undesired behavior, because it allows to forge valid kernel
pointers from userland. To avoid that, we don't share the key with
userland, and rather switch it in EL0<->EL1 transitions. This means that
when userland executes, a different key is loaded in APIAKey than the one
the kernel uses. For now the userland key is a fixed 128bit zero value.

The DDB stack unwinder is changed to strip the authentication code from
the pointers in lr.

Two problems are known:

* Currently the idlelwps' keys are not really secret. This is because
the RNG is not yet available when we spawn these lwps. Not overly
important, but would be nice to fix with UEFI RNG.
* The key switching in EL0<->EL1 transitions is not the most optimized
code on the planet. Instead of checking aarch64_pac_enabled, it would
be better to hot-patch the code at boot time, but there currently is
no hot-patch support on aarch64.

Tested on Qemu.
 1.6 19-Jan-2020  skrll branches: 1.6.4;
Replace the two copies of the ADDR macro with a centralised adrl macro.
The adrl name matches the one used by armasm.
 1.5 20-Dec-2019  ryo branches: 1.5.2;
Add a speculation barrier after the 'eret'.

Some aarch64 cpus speculatively execute instructions after 'eret',
and this potentiates side-channel attack.

from
https://github.com/torvalds/linux/commit/679db70801da9fda91d26caf13bf5b5ccc74e8e8
 1.4 05-Aug-2019  joerg Don't define register replacements when targetting 32bit ARM.
 1.3 17-Jul-2018  christos branches: 1.3.4;
centralize fp,lr definitions
 1.2 01-Apr-2018  ryo branches: 1.2.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.2 28-Jul-2018  pgoyette Sync with HEAD
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file asm.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2.2.3 21-Apr-2020  martin Sync with HEAD
 1.2.2.2 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.2.2.1 10-Jun-2019  christos Sync with HEAD
 1.3.4.1 24-Dec-2019  martin Pull up following revision(s) (requested by ryo in ticket #574):

sys/arch/aarch64/include/asm.h: revision 1.5
sys/arch/aarch64/aarch64/cpuswitch.S: revision 1.13

Add a speculation barrier after the 'eret'.

Some aarch64 cpus speculatively execute instructions after 'eret',
and this potentiates side-channel attack.

from
https://github.com/torvalds/linux/commit/679db70801da9fda91d26caf13bf5b5ccc74e8e8
 1.5.2.1 25-Jan-2020  ad Sync with head.
 1.6.4.1 20-Apr-2020  bouyer Sync with HEAD
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file bswap.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file bus_defs.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.3 10-Aug-2019  skrll Really provide bus_funcs.h
 1.2 01-Apr-2018  ryo branches: 1.2.2; 1.2.6;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file bus_funcs.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2.6.1 13-Aug-2019  martin Pull up following revision(s) (requested by skrll in ticket #53):

sys/arch/aarch64/include/bus_funcs.h: revision 1.3

Really provide bus_funcs.h
 1.2.2.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.4 17-Jan-2017  rin avoid conversion warnings
 1.3 29-Oct-2014  dennis branches: 1.3.2; 1.3.4; 1.3.6;
Correct 32 and 64 bit byte swap inlines
 1.2 11-Aug-2014  matt branches: 1.2.4;
Use %x/%w as appropriate.
 1.1 10-Aug-2014  matt Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.2.4.3 03-Dec-2017  jdolecek update from HEAD
 1.2.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.4.1 11-Aug-2014  tls file byte_swap.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.3.6.1 21-Apr-2017  bouyer Sync with HEAD
 1.3.4.1 20-Mar-2017  pgoyette Sync with HEAD
 1.3.2.1 05-Feb-2017  skrll Sync with HEAD
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file cdefs.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.53 30-Dec-2024  jmcneill arm64: Enable support for low power idle CPU states on ACPI platforms.

The ACPI CPU driver parses the _LPI package on each CPU and builds a
table of supported low power states. A custom cpu_idle() implementation
is registered that uses the time previously spent idle to select an
entry method for low power on the next idle entry.

A boot option, "nolpi", can be used to ignore _LPI and use the normal
WFI idle method.

This decreases the battery discharge rate on my Snapdragon X1E laptop from
~17W to ~10W when idle.
 1.52 10-Dec-2024  jmcneill fix 32-bit arm builds
 1.51 10-Aug-2024  riastradh aarch64: Count RNDRRS failure events and add dtrace probe.

PR port-arm/58572: aarch64 RNDRRS failures should be evcounted and
dtraced
 1.50 09-May-2024  pho branches: 1.50.2;
port-arm/58194: Resurrect vmt(4) from bitrot

On this architecture vmt(4) used to search for a node "/hypervisor" in the
FDT and probed the VMware hypervisor call only when the node was
found. However, things appear to have changed and VMware no longer provides
the FDT node.

Since vmt(4) doesn't actually need to read anything from FDT, and the
hypervisor call logically resides in virtual CPUs themselves, it would be
better to attach it directly to cpu, just like how it's probed on x86.
 1.49 25-Feb-2023  riastradh aarch64: curcpu() audit.

Sprinkle KASSERT (or KDASSERT in hot paths) for kpreempt_disabled()
when we use curcpu() and it's not immediately obvious that the caller
has preemption disabled but closer scrutiny suggests the caller has.

Note unsafe curcpu()s for syscall event counting. Not sure this is
worth changing.

Possible bugs fixed:

- cpu_irq and cpu_fiq could be preempted while trying to run softints
on this CPU.

- data_abort_handler might incorrectly think it was invoked in
interrupt context when it was only preempted and migrated to
another CPU.

- pmap_fault_fixup might report the wrong CPU logs.

(However, we don't currently run with kpreemption on aarch64, so
these are not yet real bugs fixed except if you patch it to build
with __HAVE_PREEMPTION.)
 1.48 03-Nov-2022  skrll branches: 1.48.2;
Provide MI PMAP support on AARCH64
 1.47 25-Jun-2022  jmcneill Remove GIC_SPLFUNCS.
 1.46 25-Jun-2022  jmcneill pic: Update ci_cpl in pic_set_priority callback.

Not all ICs need interrupts disabled to update the priority. DAIF accesses
are not cheap, so push the update of ci_cpl from pic_set_priority to the
IC's pic_set_priority callback, and let the IC driver determine whether
or not it needs interrupts disabled.
 1.45 02-Nov-2021  ryo In order to prevent _mcount() from being recursively called when built with COPTS=-O0,
sprinkle `__always_inline' to make _mcount() be generated as a single function.
 1.44 01-Nov-2021  skrll Fix a last minute rebase/merge botch so that the cpu_hatch commit actually
works.
 1.43 31-Oct-2021  skrll Rework Arm (32bit and 64bit) AP startup so that cpu_hatch doesn't sleep.

The AP initialisation code in cpu_init_secondary_processor will read and
initialise the required system registers and state for the BP to attach
and report.

Rework the interrupt handler code for this new sequence. Thankfully,
this removes a bunch of code for bcm2836mp.

The VFP detection handler on <= armv7 relies on the global undefined
handler being in place until the BP attaches vfp. That is, after the
APs have been spun up.

gicv3_its.c has a serialisation issue which is protected against in
the gicv3_its_cpu_init, which is called from cpu_hatch, with a spin
lock. The serialisation issue needs addressing more completely.

Tested on RPI3, Apple M1, QEMU, and lx2k

Fixes PR port-arm/56264:
diagnostic assertion "l->l_stat == LSONPROC" failed on RPI3
 1.42 31-Oct-2021  skrll Annotate some cpu_info members
 1.41 26-Oct-2021  skrll Add a comment and adjust whitespace to match style in this file
 1.40 10-Oct-2021  skrll Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>
 1.39 18-Sep-2021  jmcneill gic_splx: performance optimizations

Avoid any kind of register access (DAIF, PMR, etc), barriers, and atomic
operations in the common case where no interrupt fires between spl being
raised and lowered.

This introduces a per-CPU return address (ci_splx_restart) used by the
vector handler to restart a sequence in splx that compares the new ipl
with the per-CPU hardware priority state stored in ci_hwpl.
 1.38 14-Aug-2021  ryo Improved the performance of kernel profiling on MULTIPROCESSOR, and possible to get profiling data for each CPU.

In the current implementation, locks are acquired at the entrance of the mcount
internal function, so the higher the number of cores, the more lock conflict
occurs, making profiling performance in a MULTIPROCESSOR environment unusable
and slow. Profiling buffers has been changed to be reserved for each CPU,
improving profiling performance in MP by several to several dozen times.

- Eliminated cpu_simple_lock in mcount internal function, using per-CPU buffers.
- Add ci_gmon member to struct cpu_info of each MP arch.
- Add kern.profiling.percpu node in sysctl tree.
- Add new -c <cpuid> option to kgmon(8) to specify the cpuid, like openbsd.
For compatibility, if the -c option is not specified, the entire system can be
operated as before, and the -p option will get the total profiling data for
all CPUs.
 1.37 08-Aug-2021  skrll Re-apply

Move 'struct pic_pending' from percpu to struct cpu_info. Saves a few
instructions in splx.

There is(/was) no need to use atomic operations on the percpu / cpu_info
members, so don't.

Finally removng the use of percpu should help avoid problems with "late"
attaching cpus.
 1.36 29-May-2021  skrll Deal with the pmap limitation of maxproc in a more complete way and
recognise CPUs with only 8bit ASIDs.
 1.35 29-May-2021  skrll Sort includes. NFCI.
 1.34 27-Mar-2021  jmcneill branches: 1.34.2; 1.34.4;
Revert recent pic optimizations until I have more time to work on this.
 1.33 21-Feb-2021  jmcneill branches: 1.33.2;
Add cpu_dosoftints_ci(). Like cpu_dosoftints(), but takes a cpu_info ptr
so we can avoid the extra tpidr_el1 access if cpu_info is already known.
 1.32 21-Feb-2021  jmcneill Keep current hardware priority value in struct cpu_info and use it instead
of reading icc_pmr_el1 in gicv3_set_priority.
 1.31 20-Feb-2021  jmcneill Move 'struct pic_pending' from percpu to struct cpu_info. Saves a few
instructions in splx.
 1.30 07-Dec-2020  jmcneill ACPI Processor UID is 32-bits (ci_acpiid).
 1.29 21-Nov-2020  jmcneill Add a per-CPU event counter that counts every time an interrupt handler is
preempted by a higher priority interrupt.
 1.28 01-Oct-2020  ryo branches: 1.28.2;
fix build error with LLVM
 1.27 14-Sep-2020  ryo PID_MAX is just an initial value (soft maximum). Don't use it for CTASSERT.
defined __HAVE_CPU_MAXPROC to use function cpu_maxproc().

pointed out by mrg@, thanks.
 1.26 12-Aug-2020  skrll Part II of ad's aarch64 performance improvements (cpu_switch.S bugs are
all mine)

- Use tpidr_el1 to hold curlwp and not curcpu, because curlwp is accessed
much more often by MI code. It also makes curlwp preemption safe and
allows aarch64_curlwp() to be a const function (curcpu must be volatile).

- Make ASTs operate per-LWP rather than per-CPU, otherwise sometimes LWPs
can see spurious ASTs (which doesn't cause a problem, it just means some
time may be wasted).

- Use plain stores to set/clear ASTs. Make sure ASTs are always set on the
same CPU as the target LWP, and delivered via IPI if posted from a remote
CPU so that they are resolved quickly.

- Add some cache line padding to struct cpu_info, to match x86.

- Add a memory barrier in a couple of places where ci_curlwp is set. This
is needed whenever an LWP that is resuming on the CPU could hold an
adaptive mutex. The barrier needs to drain the CPU's store buffer, so
that the update to ci_curlwp becomes globally visible before the LWP can
resume and call mutex_exit(). By my reading of the ARM docs it looks like
the instruction I used will do the right thing, but I'm not 100% sure.
 1.25 01-Jul-2020  ryo - On some systems with a different cache line size (and DIC,IDC) per CPU, trap "mrs Xt,ctr_el0" instruction
to return the minimum cache line size of the system to userland.
- add CLIDR_EL1 and CTR_EL0 to struct aarch64_sysctl_cpu_id.

On most systems, cache line size is the same for all CPUs, so this mechanism won't be required.
Rather, this is primarily for errata support, which will be committed later.
 1.24 01-Jul-2020  ryo Switch the Icache sync operation to the necessary and sufficient one according to the CTR_EL0.DIC and CTR_EL0.IDC flags.

If CTR_EL0.DIC=1, Icache invalidation is not required.
If CTR_EL0.IDC=1, Dcache clean before Icache invalidation is not required.
CLIDR_EL1.LoC is 0, or CLIDR_EL1.LoUIS and CLIDR_EL1.LoUU are 0, Dcache clean is not required as well.

SEE ALSO ARMARM, "CTR_EL0 Cache Type Register", and "CLIDR_EL1 Cache Level ID Register"
 1.23 29-Jun-2020  riastradh Draft fpu_kern_enter/leave on aarch64.
 1.22 10-Mar-2020  christos protect curcpu/curlwp from _KMEMUSER
 1.21 15-Feb-2020  skrll Various updates and improvements to cpu start up on arm/aarch64

- start sharing more code around the AP startup messaging.
- call arm_cpu_topology_set early so that ci_core_id is available for
drivers, e.g. bcm2835_intr.c
- both arm and aarch64 now have
- a static cpu_info_store array
- the same arm_cpu_{hatched,mbox}
 1.20 12-Feb-2020  riastradh Define the MULTIPROCESSOR cpu_number() for modules too.

Modules should work whether the main kernel is multiprocessor or not.
In particular, dtrace should not think cpu_number() is 0 while
cpu_index(curcpu()) and curcpu()->ci_index are nonzero, leading to
rather spectacularly bogus results...
 1.19 15-Jan-2020  mrg port the arm64 cpu topology setup for big.little to arm.

rename arm64 cpu_do_topology() to arm_cpu_do_topology() and
call it from both arm cpu_attach().

replace both aarch64_set_topology() inline code in arm
cpu_attach() with new arm_cpu_do_topology(), which is called
by the arm64 locore as well (possibly not needed, which would
allow it to become static.)

not yet tested on a real big.little armv7 system. tested
on rockpro64 and pinebook pro.
 1.18 12-Jan-2020  mrg provide some semblance of valid cpu topology for big.little systems.

while attaching cpus, if the FDT provides "capacity-dmips-mhz" track
the fastest set, and call cpu_topology_set() with slow=true for any
cpus that are not the fastest.

bug fix for cpu_topology_set(): actually set ci_is_slow for slow cpus.

with this change, and -current's recent scheduler changes, this means
that long running processes run on the faster cores. on RK3399 based
systems, i am seeing 20-50% speed ups for many tasks.


XXX: all this can be made common with armv7 big.little.
 1.17 05-Jan-2020  ad branches: 1.17.2;
Give aarch64 a preemption safe cpu_intr_p().
 1.16 02-Dec-2019  ad + ci_onproc
 1.15 21-Nov-2019  ad mi_userret(): take care of calling preempt(), set spc_curpriority directly,
and remove MD code that does the same.
 1.14 19-Oct-2019  jmcneill Increase aarch64 MAXCPUS to 256.
 1.13 21-Dec-2018  ryo branches: 1.13.4;
- add workaround for Cavium ThunderX errata 27456.
- add cpufuncs table in cpu_info. each cpu clusters may have different erratum. (e.g. big.LITTLE)
 1.12 24-Nov-2018  skrll Provide a LWP_PC for Taylor
 1.11 20-Nov-2018  mrg rewrite the CPU identification on arm64:

- publish per-cpu data
- publish a whole bunch of info in struct aarch64_sysctl_cpu_id
instead of various individual nodes (there are 16 total.)
- add MIDR extractor bits
- define ARMv8.2-A id_aa64mmfr2_el1 and id_aa64zfr0_el1 regs,
but avoid using them until we make sure they exist. (these
members are added to aarch64_sysctl_cpu_id to avoid future
compat issues.)

the arm32 and aarch32 version of these need to be adjusted as
well (and aarch32 data published at all.) still trying to
work out how to make the same userland binary running on a
real arm32 or an aarch32 system can work sanely here.

ok ryo@.
 1.10 18-Oct-2018  skrll Provide generic start code that assumes the MMU is off and caches are
disabled as per the linux booting protocol for ARMv6 and ARMv7 boards.
u-boot image type should be changed to 'linux' for correct behaviour.

The new start code builds a minimal "bootstrap" L1PT with cached access
disabled and uses the same table for all processors. AP startup is
performed in less steps and more code is written in C.

The bootstrap tables and stack are placed into an (orphaned) section
"_init_memory" which is given to uvm when it is no longer used.

Various kernels have been converted to use this code and tested. Some
boards were provided by TNF. Thanks!

The GENERIC kernel now boots on boards using the TEGRA, SUNXI and EXYNOS
kernels. The GENERIC kernel will also work on RPI2 using u-boot.

Thanks to martin@ and aymeric@ for testing on parallella and nanosoc
respectively
 1.9 12-Oct-2018  jmcneill Add ACPI Processor Unique ID (ci_acpiid) to struct cpu_info, required by
ACPI subsystem.
 1.8 10-Sep-2018  ryo cleanup aarch64 mpstart and fdt bootstrap
* arm_cpu_hatch_arg is a bad idea. avoid serializing CPU startup, and eliminate arm_cpu_hatch_arg.
in mpstart, resolve own cpu index using array of cpu_mpidr[] (aarch64)
* add support fdt enable-method "spin-table"
* add support fdt enable-method "brcm,bcm2836-smp" (for 32bit RaspberryPi)
* use arm_fdt_cpu_bootstrap() instead of psci_fdt_bootstrap()
* rename "arm/fdt/psci_fdt.h" to "arm/fdt/psci_fdtvar.h" because of conflict of include file for needs-flag
* add devmap for cpu spin-table of raspberrypi3/aarch64
* no need to force hatch APs for raspberrypi3/arm32 ifndef MULTIPROCESSOR.
* fix to work pmap_extract(kerneltext/data/bss) even if before calling pmap_bootstrap

idea to use cpu_mpidr[] by jmcneill@. reviewd by skrll@. thanks.
 1.7 26-Aug-2018  ryo add support multiple cpu clusters.
* pass cpu index as an argument to secondary processors when hatching.
* keep cpu cache confituration per cpu clusters.

Hello big.LITTLE!
 1.6 08-Aug-2018  jmcneill Add fields for per-cpu GICv3 state
 1.5 23-Jul-2018  ryo rather than using flags to resolve nested locks, reserve pool_cache before locking.
 1.4 21-Jul-2018  ryo * avoid deadlock. mutex_owned() works only for adaptive lock, therefore we cannot use it for spinlock...
* add more NULL check
* clear pte when pmap_enter() fails
 1.3 09-Jul-2018  ryo add MULTIPROCESSOR support
 1.2 01-Apr-2018  ryo branches: 1.2.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.7 26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.1.28.6 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.1.28.5 20-Oct-2018  pgoyette Sync with head
 1.1.28.4 30-Sep-2018  pgoyette Ssync with HEAD
 1.1.28.3 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.1.28.2 28-Jul-2018  pgoyette Sync with HEAD
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file cpu.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2.2.2 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.2.2.1 10-Jun-2019  christos Sync with HEAD
 1.13.4.2 12-Feb-2020  martin Pull up following revision(s) (requested by riastradh in ticket #701):

external/cddl/osnet/dev/dtrace/aarch64/dtrace_isa.c: revision 1.2
external/cddl/osnet/dist/lib/libdtrace/common/dt_open.c: revision 1.17
external/cddl/osnet/dist/lib/libdtrace/common/dt_module.c: revision 1.18
sys/modules/cyclic/Makefile: revision 1.5
external/cddl/osnet/dev/dtrace/aarch64/dtrace_subr.c: revision 1.2
external/cddl/osnet/dev/dtrace/aarch64/dtrace_subr.c: revision 1.3
sys/arch/aarch64/aarch64/vectors.S: revision 1.10
external/cddl/osnet/dev/fbt/aarch64/fbt_isa.c: revision 1.2
external/cddl/osnet/dev/fbt/aarch64/fbt_isa.c: revision 1.3
external/cddl/osnet/dev/fbt/aarch64/fbt_isa.c: revision 1.4
external/cddl/osnet/dev/fbt/aarch64/fbt_isa.c: revision 1.5
external/cddl/osnet/dev/fbt/aarch64/fbt_isa.c: revision 1.6
sys/arch/aarch64/include/cpu.h: revision 1.20
external/cddl/osnet/dist/lib/libdtrace/common/dt_impl.h: revision 1.9

Create a buffer space of 512 bytes before the trapframe.

dtrace fbt needs enough space to emulate an

stp x29, x30, [sp,#-FRAMESIZE]!

instruction in a function prologue. In the aarch64 instruction
encoding, FRAMESIZE can be as large as 512 bytes, so reserve this
much space when KDTRACE_HOOKS is enabled.

Use db_write_bytes to overwrite kernel text.

Tidy up a bit. No functional change intended.

aarch64 fbt_invop doesn't actually use the argument, but it would
make more sense for it to be the return value and/or first argument
register. Certainly it's not `eax'!

Tidy up a bit: don't set things we won't use; assert nonzeroness.

Use /dev/ksyms, not /netbsd, for the running kernel's symbols.

Teach dtrace about el1_trap_exit frames on aarch64.

Implement dtrace_getarg and dtrace_getreg while here.

Count the number of artificial frames in aarch64 fbt probe correctly.

Change the address ranges that aarch64 considers toxic for dtrace.
`Toxic' means dtrace forbids D scripts from even attempting to read
or write at them.

Previously we considered [0, VM_MIN_KERNEL_ADDRESS) toxic, but
VM_MIN_KERNEL_ADDRESS is only the minimum address of the kernel map;
the direct-mapped region lies below it, and with PMAP_MAP_POOLPAGE we
allocate virtual pages for pool backing directly from physical pages
through the direct-mapped region. Also, this did not consider I/O
mappings to be toxic, which they probably should be.

Instead, treat:

[0, AARCH64_KSEG_START)
and
[VM_KERNEL_IO_ADDRESS, 0xfff...ff)

as toxic. (The upper bound for 0xfff...ff ought to be inclusive, not
exclusive, but I think we'll need another mechanism for expressing
that to dtrace!)

Switch from db_write_bytes to using direct-mapping.

This way there's no dependency on ddb.

Define the MULTIPROCESSOR cpu_number() for modules too.
Modules should work whether the main kernel is multiprocessor or not.
In particular, dtrace should not think cpu_number() is 0 while
cpu_index(curcpu()) and curcpu()->ci_index are nonzero, leading to
rather spectacularly bogus results...

cyclic.kmod needs -Wno-sign-compare for aarch64 CPU_INFO_FOREACH.
Provisional workaround; feel free to fix.
 1.13.4.1 23-Oct-2019  martin Pull up following revision(s) (requested by jmcneill in ticket #359):

sys/arch/aarch64/aarch64/locore.S: revision 1.42
sys/arch/aarch64/aarch64/locore.S: revision 1.43
sys/arch/aarch64/aarch64/locore.S: revision 1.44
sys/arch/arm/fdt/cpu_fdt.c: revision 1.28
sys/arch/aarch64/include/cpu.h: revision 1.14
sys/arch/aarch64/include/param.h: revision 1.12
sys/arch/arm/arm32/cpu.c: revision 1.133
sys/arch/arm/arm32/cpu.c: revision 1.134
sys/arch/arm/include/cpu.h: revision 1.101
sys/arch/arm/acpi/cpu_acpi.c: revision 1.7
sys/arch/aarch64/aarch64/cpu.c: revision 1.23
sys/arch/aarch64/aarch64/cpu.c: revision 1.24
sys/arch/aarch64/aarch64/cpu.c: revision 1.25

Increase aarch64 MAXCPUS to 256.

-

Invalidate dcache before polling AP hatched status

-

Avoid overlap between BP and last AP stack. AP stacks are now in order of
increasing address order.

Spotted by and idea from mlelstv.

-

Use separate cacheline aligned arrays for mbox and hatched as before.

-

cpu_hatched_p only for MULTIPROCESSOR
 1.17.2.2 29-Feb-2020  ad Sync with head.
 1.17.2.1 17-Jan-2020  ad Sync with head.
 1.28.2.2 03-Apr-2021  thorpej Sync with HEAD.
 1.28.2.1 14-Dec-2020  thorpej Sync w/ HEAD.
 1.33.2.1 03-Apr-2021  thorpej Sync with HEAD.
 1.34.4.1 31-May-2021  cjep sync with head
 1.34.2.1 17-Jun-2021  thorpej Sync w/ HEAD.
 1.48.2.1 13-Oct-2024  martin Pull up following revision(s) (requested by riastradh in ticket #955):

sys/arch/aarch64/aarch64/cpu.c: revision 1.78
sys/arch/aarch64/include/cpu.h: revision 1.51

aarch64: Count RNDRRS failure events and add dtrace probe.

PR port-arm/58572: aarch64 RNDRRS failures should be evcounted and
dtraced
 1.50.2.1 02-Aug-2025  perseant Sync with HEAD
 1.2 22-Feb-2021  ryo PR/56002: aarch64 has a true 64bit CPU cycle counter, we will use it.

This fix solves PR/56002 on aarch64, but this problems can occur on
all other architectures where cpu_counter() is 32bit.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.42;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.42.1 03-Apr-2021  thorpej Sync with HEAD.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file cpu_counter.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.28 30-Dec-2024  jmcneill aarch64: Allow for alternate cpu_idle() implementations
 1.27 07-Feb-2024  msaitoh branches: 1.27.2;
Remove ryo@'s mail addresses.
 1.26 20-Apr-2023  skrll Provide a shared pmap_devmap implementation and convert all pmap_devmap
arrays to use DEVMAP_ENTRY{,_END}
 1.25 10-Sep-2022  rillig fix misspellings of 'available' and nearby typos
 1.24 20-Jul-2022  riastradh aarch64: Make cpufunc.h includable without sys/cpu.h first.
 1.23 31-Jan-2022  ryo add support Hardware updates to Access flag and Dirty state (FEAT_HAFDBS)

- The DBM bit of the PTE is now used to determine if it is writable, and
the AF bit is treated entirely as a reference bit. A valid PTE is always
treated as readable. There can be no valid PTE that is not readable.
- LX_BLKPAG_OS_{READ,WRITE} are used only for debugging purposes,
and has been superseded by LX_BLKPAG_AF and LX_BLKPAG_DBM.
- Improve comment

The need for reference/modify emulation has been eliminated,
and access/permission faults have been reduced, however,
there has been little change in overall performance.
 1.22 31-Oct-2021  skrll Rework Arm (32bit and 64bit) AP startup so that cpu_hatch doesn't sleep.

The AP initialisation code in cpu_init_secondary_processor will read and
initialise the required system registers and state for the BP to attach
and report.

Rework the interrupt handler code for this new sequence. Thankfully,
this removes a bunch of code for bcm2836mp.

The VFP detection handler on <= armv7 relies on the global undefined
handler being in place until the BP attaches vfp. That is, after the
APs have been spun up.

gicv3_its.c has a serialisation issue which is protected against in
the gicv3_its_cpu_init, which is called from cpu_hatch, with a spin
lock. The serialisation issue needs addressing more completely.

Tested on RPI3, Apple M1, QEMU, and lx2k

Fixes PR port-arm/56264:
diagnostic assertion "l->l_stat == LSONPROC" failed on RPI3
 1.21 23-Oct-2021  skrll Whitespace
 1.20 27-May-2021  ryo fix build error with options ARMV85_BTI
 1.19 04-Dec-2020  skrll branches: 1.19.4; 1.19.6;
Remove unnecessary casts
 1.18 03-Aug-2020  ryo branches: 1.18.2;
Implement MD ucas(9) (__HAVE_UCAS_FULL)
 1.17 02-Aug-2020  maxv Add support for Privileged Access Never (ARMv8.1-PAN).

PAN provides the same functionality as SMAP on x86: it forbids kernel
access to userland pages when PSTATE.PAN=1, and allows such accesses when
PSTATE.PAN=0.

We clear SCTLR_SPAN, to guarantee that PAN=1 each time the kernel is
entered. We catch PAN faults and panic right away without further
processing. In copyin, copyout, etc, we temporarily authorize access to
userland pages.

PAN is a very useful exploit mitigation. Reviewed by ryo@, thanks. Tested
on Qemu. Enabled by default.
 1.16 01-Jul-2020  ryo Switch the Icache sync operation to the necessary and sufficient one according to the CTR_EL0.DIC and CTR_EL0.IDC flags.

If CTR_EL0.DIC=1, Icache invalidation is not required.
If CTR_EL0.IDC=1, Dcache clean before Icache invalidation is not required.
CLIDR_EL1.LoC is 0, or CLIDR_EL1.LoUIS and CLIDR_EL1.LoUU are 0, Dcache clean is not required as well.

SEE ALSO ARMARM, "CTR_EL0 Cache Type Register", and "CLIDR_EL1 Cache Level ID Register"
 1.15 25-May-2020  ryo cache information can be detected correctly on newer CPUs

- add VPIPT cache type
- adapt to 64-bit CCSIDR (ARMv8.3-CCIDX)
- CCSIDR:[WT,WB,PA,WA] are deprecated
- show number of cache lines when attaching cpu
 1.14 15-May-2020  ryo SCTLR_EnIA should be enabled in the caller(locore).

For some reason, gcc make aarch64_pac_init() function non-leaf, and it uses paciasp/autiasp.
 1.13 13-May-2020  ryo - move aarch64 addressspace macros from pmap.h to cpufunc.h
- rename ptr_strip_pac() to aarch64_strip_pac()
 1.12 12-Apr-2020  maxv Add support for Pointer Authentication (PAC).

We use the "pac-ret" option, to sign the return instruction pointer on
function entry, and authenticate it on function exit. This acts as a
mitigation against ROP.

The authentication uses a per-lwp (secret) I-A key stored in the 128bit
APIAKey register and part of the lwp context. During lwp creation, the
kernel generates a random key, and during context switches, it installs
the key of the target lwp on the CPU.

Userland cannot read the APIAKey register directly. However, it can sign
its pointers with it, because the register is architecturally shared
between userland and the kernel. Although part of the CPU design, it is
a bit of an undesired behavior, because it allows to forge valid kernel
pointers from userland. To avoid that, we don't share the key with
userland, and rather switch it in EL0<->EL1 transitions. This means that
when userland executes, a different key is loaded in APIAKey than the one
the kernel uses. For now the userland key is a fixed 128bit zero value.

The DDB stack unwinder is changed to strip the authentication code from
the pointers in lr.

Two problems are known:

* Currently the idlelwps' keys are not really secret. This is because
the RNG is not yet available when we spawn these lwps. Not overly
important, but would be nice to fix with UEFI RNG.
* The key switching in EL0<->EL1 transitions is not the most optimized
code on the planet. Instead of checking aarch64_pac_enabled, it would
be better to hot-patch the code at boot time, but there currently is
no hot-patch support on aarch64.

Tested on Qemu.
 1.11 15-Jan-2020  mrg branches: 1.11.4;
port the arm64 cpu topology setup for big.little to arm.

rename arm64 cpu_do_topology() to arm_cpu_do_topology() and
call it from both arm cpu_attach().

replace both aarch64_set_topology() inline code in arm
cpu_attach() with new arm_cpu_do_topology(), which is called
by the arm64 locore as well (possibly not needed, which would
allow it to become static.)

not yet tested on a real big.little armv7 system. tested
on rockpro64 and pinebook pro.
 1.10 12-Jan-2020  mrg provide some semblance of valid cpu topology for big.little systems.

while attaching cpus, if the FDT provides "capacity-dmips-mhz" track
the fastest set, and call cpu_topology_set() with slow=true for any
cpus that are not the fastest.

bug fix for cpu_topology_set(): actually set ci_is_slow for slow cpus.

with this change, and -current's recent scheduler changes, this means
that long running processes run on the faster cores. on RK3399 based
systems, i am seeing 20-50% speed ups for many tasks.


XXX: all this can be made common with armv7 big.little.
 1.9 19-Dec-2019  ryo branches: 1.9.2;
aarch64_cache_info[] is not global
 1.8 22-Nov-2019  mlelstv Make cache operations available early.
 1.7 13-Sep-2019  ryo In pmap_devmap_bootstrap(), cpu_earlydevice_va_p() must not return true until *all* devmap tables have been enabled.
console mapping may be present in the last table.
 1.6 07-Sep-2019  ryo add checking status of MMU and devmap to make _platform_early_putchar() available at all times.
 1.5 21-Dec-2018  ryo - add workaround for Cavium ThunderX errata 27456.
- add cpufuncs table in cpu_info. each cpu clusters may have different erratum. (e.g. big.LITTLE)
 1.4 15-Dec-2018  alnsn Add missing include for device_t declaration.
 1.3 26-Aug-2018  ryo add support multiple cpu clusters.
* pass cpu index as an argument to secondary processors when hatching.
* keep cpu cache confituration per cpu clusters.

Hello big.LITTLE!
 1.2 23-Jul-2018  ryo * fix icache invalidations.
* "ic ivau" (aarch64_icache_sync_range) with VA generates permission fault in some situations, therefore use KSEG address for now.
 1.1 01-Apr-2018  ryo branches: 1.1.2; 1.1.4;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1.4.4 21-Apr-2020  martin Sync with HEAD
 1.1.4.3 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.1.4.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.1.4.1 10-Jun-2019  christos Sync with HEAD
 1.1.2.5 26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.1.2.4 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.1.2.3 28-Jul-2018  pgoyette Sync with HEAD
 1.1.2.2 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.2.1 01-Apr-2018  pgoyette file cpufunc.h was added on branch pgoyette-compat on 2018-04-07 04:12:11 +0000
 1.9.2.1 17-Jan-2020  ad Sync with head.
 1.11.4.1 20-Apr-2020  bouyer Sync with HEAD
 1.18.2.1 14-Dec-2020  thorpej Sync w/ HEAD.
 1.19.6.1 31-May-2021  cjep sync with head
 1.19.4.1 17-Jun-2021  thorpej Sync w/ HEAD.
 1.27.2.1 02-Aug-2025  perseant Sync with HEAD
 1.16 31-Oct-2021  skrll Fix crash(8) build
 1.15 31-Oct-2021  skrll Rework Arm (32bit and 64bit) AP startup so that cpu_hatch doesn't sleep.

The AP initialisation code in cpu_init_secondary_processor will read and
initialise the required system registers and state for the BP to attach
and report.

Rework the interrupt handler code for this new sequence. Thankfully,
this removes a bunch of code for bcm2836mp.

The VFP detection handler on <= armv7 relies on the global undefined
handler being in place until the BP attaches vfp. That is, after the
APs have been spun up.

gicv3_its.c has a serialisation issue which is protected against in
the gicv3_its_cpu_init, which is called from cpu_hatch, with a spin
lock. The serialisation issue needs addressing more completely.

Tested on RPI3, Apple M1, QEMU, and lx2k

Fixes PR port-arm/56264:
diagnostic assertion "l->l_stat == LSONPROC" failed on RPI3
 1.14 30-Apr-2021  skrll Make the ddb for pmap / pte information pmap agnostic
 1.13 11-Mar-2021  ryo branches: 1.13.4;
- fixed a problem where hardware {break,watch}points other than #0 could not be cleared
- hardware {break,watch}point addresses are now strictly checked
 1.12 09-Mar-2021  ryo Add support hardware breakpoint and watchpoint again.

Limited support for hardware watchpoint has been available for some time, but it
has not been working properly. In addition, it stopped working at the time of
the PTRACE support commit on 2018-12-13. This has been fixed to work correctly,
and also fixed to be practical by sharing hardware watchpoints and breakpoints
between CPUs on MULTIPROCESSOR.

Also fixed a bug that causes a malfunction when switching CPUs with
"machine cpu N" when entering ddb mode from other than cpu_Debugger().

I have confirmed that the CPU can be switched by "machine cpu N" and return from
ddb properly in each case where ddb is called triggered by ddb break/watchpoint,
hardware break/watchpoint, and cpu_Debugger().
 1.11 14-Sep-2020  ryo branches: 1.11.2;
sprinkle LE32TOH to fetch instructions on aarch64eb
 1.10 08-Jul-2020  ryo Determination of A64,A32,T32 for disasm is now done in strrdisasm() instead of the caller.
correctly disassemble by processor state if defined DEBUG_DUMP_ON_USERFAULT or DEBUG_DDB_ON_USERFAULT.
 1.9 22-May-2020  ryo fix to do backtrace properly for running LWPs and cpu_lwp_fork().
when dump of pcb_tf, only the switchframe part is now displayed instead of the whole trapframe.
 1.8 12-Oct-2018  ryo add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.
 1.7 15-Sep-2018  jakllsch make kernel-groveling crash(8) work on aarch64
 1.6 17-Jul-2018  ryo use panic() instead of some printf to show fault status.
useful for ddb "show panic" command.
 1.5 28-Apr-2018  ryo branches: 1.5.2;
Oops, my previous commit is totally wrong. recast mask/pattern list.
pointed out by David Binderman in PR/53224, thanks.
 1.4 27-Apr-2018  ryo remove suspicious compare, and cleanup complex conditionals.
pointed out PR/53159 by dcb314, thanks.
 1.3 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.2 11-Jan-2016  skrll branches: 1.2.16;
PR port-arm/50641: src/sys/arch/aarch64/include/db_machdep.h:67: possible bad if test ?

Fix the bl instruction test.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.6;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.6.1 19-Mar-2016  skrll Sync with HEAD
 1.1.4.3 03-Dec-2017  jdolecek update from HEAD
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file db_machdep.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2.16.5 20-Oct-2018  pgoyette Sync with head
 1.2.16.4 30-Sep-2018  pgoyette Ssync with HEAD
 1.2.16.3 28-Jul-2018  pgoyette Sync with HEAD
 1.2.16.2 02-May-2018  pgoyette Synch with HEAD
 1.2.16.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.5.2.1 10-Jun-2019  christos Sync with HEAD
 1.11.2.1 03-Apr-2021  thorpej Sync with HEAD.
 1.13.4.1 13-May-2021  thorpej Sync with HEAD.
 1.2 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file disklabel.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.5 30-May-2022  jkoshy Use the ABI value for 'R_AARCH64_TLSLD_LDST128_DTPREL_LO12_NC'.
 1.4 12-Oct-2018  ryo add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.
 1.3 15-Aug-2018  ryo fix relocation type. 277 is R_AARCH64_ADD_ABS_LO12_NC
 1.2 06-Nov-2017  christos branches: 1.2.2; 1.2.4;
Cleanup and clarify the ELFSIZE mess:

We now have 2 variables automatically set in elf_machdep.h:

ARCH_ELFSIZE: the size for userland binaries
KERN_ELFSIZE: the size for the kernel binaries

DB_ELFSIZE has been deleted and KERN_ELFSIZE should have always the
same values DB_ELFSIZE used to have.

In sys/exec_elf.h, if ELFSIZE is not set, it is set to KERN_ELFSIZE
for the kernel and ARCH_ELFSIZE for userland. These defaults should
eliminate the need for most manual ELFSIZE setting.
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.3 03-Dec-2017  jdolecek update from HEAD
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file elf_machdep.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2.4.1 10-Jun-2019  christos Sync with HEAD
 1.2.2.2 20-Oct-2018  pgoyette Sync with head
 1.2.2.1 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file endian.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file endian_machdep.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file fenv.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file float.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2 29-Jun-2020  riastradh Move aarch64/fpu.h to arm/fpu.h.
 1.1 29-Jun-2020  riastradh Draft fpu_kern_enter/leave on aarch64.
 1.5 06-Aug-2020  ryo revert the changes of http://mail-index.netbsd.org/source-changes/2020/08/03/msg120183.html

This change is overengineered.
bus_space_{peek,poke}_N does not have to be reentrant nor available for interrupt context.

requested by skrll@
 1.4 03-Aug-2020  ryo Fix a problem in which a fault occured in an interrupt handler during copyin/copyout was erroneously detected as being occured by copyin.

- keep idepth in faultbuf and compare it to avoid unnecessary fault recovery
- make cpu_set_onfault() nestable to use bus_space_{peek,poke}()
in hardware interrupt handlers during copyin & copyout.
 1.3 03-Dec-2019  jmcneill Define lwp_trapframe() macro
 1.2 01-Apr-2018  ryo branches: 1.2.2; 1.2.6;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file frame.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2.6.1 09-Dec-2019  martin Pull up following revision(s) (requested by jmcneill in ticket #525):

external/cddl/osnet/dev/dtrace/aarch64/dtrace_isa.c: revision 1.1
distrib/sets/lists/modules/md.i386: revision 1.83
share/mk/bsd.own.mk: revision 1.1168
usr.bin/mkubootimage/mkubootimage.c: revision 1.25
sys/modules/dtrace/Makefile: revision 1.7
usr.bin/mkubootimage/mkubootimage.c: revision 1.26
sys/modules/dtrace/Makefile: revision 1.8
external/cddl/osnet/dist/lib/libdtrace/aarch64/dt_isadep.c: revision 1.2
distrib/sets/lists/modules/mi: revision 1.128
sys/arch/aarch64/include/frame.h: revision 1.3
sys/arch/evbarm/conf/mk.generic64: revision 1.4
external/cddl/osnet/dist/lib/libdtrace/common/dt_link.c: revision 1.12
sys/modules/cyclic/Makefile: revision 1.4
sys/arch/aarch64/conf/Makefile.aarch64: revision 1.16
external/cddl/osnet/dev/dtrace/aarch64/dtrace_subr.c: revision 1.1
sys/arch/aarch64/aarch64/start.S: revision 1.3
sys/arch/aarch64/aarch64/trap.c: revision 1.22
external/cddl/osnet/dev/fbt/aarch64/fbt_isa.c: revision 1.1
external/cddl/osnet/dev/dtrace/aarch64/dtrace_asm.S: revision 1.1
external/cddl/osnet/dev/fbt/aarch64/fbt_isa.h: revision 1.1
external/cddl/osnet/dev/dtrace/aarch64/regset.h: revision 1.1
external/cddl/osnet/lib/libdtrace/Makefile: revision 1.26
distrib/sets/lists/modules/md.amd64: revision 1.82
usr.bin/mkubootimage/mkubootimage.1: revision 1.13
distrib/sets/lists/modules/ad.arm: revision 1.14

Add KDTRACE_HOOKS support.

Define lwp_trapframe() macro

dtrace: add support for aarch64

Add syscall_linux back for other arm architectures (accidently removed
in previous)

Add -u flag for updating headers in place.

Fix alignment of .text section by changing load address to
0xffffffc000000000 and adding 64 bytes of padding before the entry point.

Update arm64 image header in place

Move dtrace_syscall_linux out of mi set list

Enable DTrace on aarch64

Fix signed/unsigned comparison
 1.2.2.1 08-Apr-2020  martin Merge changes from current as of 20200406
 1.3 30-Aug-2021  jmcneill Add definition for HCR_E2H bit
 1.2 29-Aug-2020  maxv Slightly clarify, and style.
 1.1 01-Apr-2018  ryo branches: 1.1.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1.2.2 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.2.1 01-Apr-2018  pgoyette file hypervisor.h was added on branch pgoyette-compat on 2018-04-07 04:12:11 +0000
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file ieee.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file ieeefp.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file int_const.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.3 13-Aug-2014  matt branches: 1.3.2;
Back out last change.
 1.2 13-Aug-2014  justin Add formatting for aarch64 as using arm ones errors for ll on 64 bit types
 1.1 10-Aug-2014  matt Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.3.2.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.3.2.1 13-Aug-2014  tls file int_fmtio.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file int_limits.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file int_mwgwtypes.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file int_types.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file intr.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2 09-Nov-2018  mrg implement dumpsys() and friends for arm64.

this is almost a direct copy of the arm code, which is simply
as the basic structures about physical memory are the same
between arm and arm64. the main change i made was to use
the direct map instead of a virtual dump page that is remapped
to whatever physical page is being dumped.

i also changed the existing cpu_kcore_hdr_t to include the
missing number of ram segments.

note that this is not a complete solution for crash dumps yet,
as the libkvm code needs some work. i'm fairly positive that
this side is correct, as i can see the data i expect to see,
but libkvm's _kvm_kvtop() function returns garbage so far.

there is no "minidump" support here yet, ala amd64, but we
probably want it eventually.


ok skrll@.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28; 1.1.30;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.30.1 10-Jun-2019  christos Sync with HEAD
 1.1.28.1 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file kcore.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file limits.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.5 24-Jul-2022  riastradh aarch64/lock.h: Need <sys/param.h> for _HARDKERNEL.

Add include guard while here.

XXX Why does this aarch64-specific file have #ifdef __aarch64__?
 1.4 26-Sep-2021  jmcneill Use the yield instruction as SPINLOCK_BACKOFF_HOOK for aarch64.
 1.3 26-Jun-2015  matt Use <sys/common_lock.h> for !__arm__
 1.2 13-Aug-2014  matt branches: 1.2.2; 1.2.4;
Use __ATOMIC_RELAXED in __cpu_simple_lock_init
 1.1 10-Aug-2014  matt Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.2.4.1 22-Sep-2015  skrll Sync with HEAD
 1.2.2.3 03-Dec-2017  jdolecek update from HEAD
 1.2.2.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.2.1 13-Aug-2014  tls file lock.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.9 01-Mar-2021  jmcneill Add DISABLE_INTERRUPT_SAVE(), like DISABLE_INTERRUPT() but also returns
the previous state.

Use DISABLE_INTERRUPT_SAVE()/ENABLE_INTERRUPT() in pic_splfuncs instead
of cpsid()/cpsie(). The difference here is the caller no longer specifies
which bits to disable and enable; on arm32 we continue to use I32_bit and
on aarch64 we now consistently toggle both IRQ and FIQ state.
 1.8 20-Feb-2021  jmcneill daif_disable: since we read bits before setting them, if the current state
matches the desired state we can skip the daif write
 1.7 07-Feb-2021  jmcneill Use reg_daif{set,clr}_write directly instead of daif_{en,dis}able for
ENABLE_INTERRUPT() and DISABLE_INTERRUPT() macros, to avoid an unnecessary
reg_daif_read().
 1.6 30-Oct-2020  skrll branches: 1.6.2;
Retire arm_[di]sb in favour of the isb() and dsb(sy) macro invocations.
 1.5 09-Jul-2018  jmcneill Include aarch64/machdep.h for arm32 compat.
 1.4 09-Jul-2018  ryo add MULTIPROCESSOR support
 1.3 01-Apr-2018  ryo branches: 1.3.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.2 16-Mar-2017  chs branches: 1.2.12;
allow pcu_save() and pcu_discard() to be called on other threads,
ptrace needs to use it that way.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.6; 1.1.10; 1.1.14;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.14.1 21-Apr-2017  bouyer Sync with HEAD
 1.1.10.1 20-Mar-2017  pgoyette Sync with HEAD
 1.1.6.1 28-Aug-2017  skrll Sync with HEAD
 1.1.4.3 03-Dec-2017  jdolecek update from HEAD
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file locore.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2.12.2 28-Jul-2018  pgoyette Sync with HEAD
 1.2.12.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.3.2.1 10-Jun-2019  christos Sync with HEAD
 1.6.2.1 03-Apr-2021  thorpej Sync with HEAD.
 1.1 30-Nov-2024  christos branches: 1.1.4;
Create a new header lwp_private.h to contain _lwp_getprivate_fast,
_lwp_gettcb_fast, _lwp_settcb and remove them from mcontext.h, so that:
1. we don't need special hacks to hide them
2. we can include <lwp.h> where needed to get the necessary prototypes
without redefining them locally.
 1.1.4.2 02-Aug-2025  perseant Sync with HEAD
 1.1.4.1 30-Nov-2024  perseant file lwp_private.h was added on branch perseant-exfatfs on 2025-08-02 05:55:20 +0000
 1.19 07-Feb-2024  msaitoh Remove ryo@'s mail addresses.
 1.18 30-Aug-2021  jmcneill Add FIQ support.
 1.17 06-Sep-2020  ryo Fix panic caused by modload. http://mail-index.netbsd.org/port-arm/2020/08/30/msg006960.html

The address space reserved for modules may not be mapped in L1-L3.
 1.16 06-Aug-2020  ryo revert the changes of http://mail-index.netbsd.org/source-changes/2020/08/03/msg120183.html

This change is overengineered.
bus_space_{peek,poke}_N does not have to be reentrant nor available for interrupt context.

requested by skrll@
 1.15 03-Aug-2020  ryo Fix a problem in which a fault occured in an interrupt handler during copyin/copyout was erroneously detected as being occured by copyin.

- keep idepth in faultbuf and compare it to avoid unnecessary fault recovery
- make cpu_set_onfault() nestable to use bus_space_{peek,poke}()
in hardware interrupt handlers during copyin & copyout.
 1.14 08-Jul-2020  ryo Determination of A64,A32,T32 for disasm is now done in strrdisasm() instead of the caller.
correctly disassemble by processor state if defined DEBUG_DUMP_ON_USERFAULT or DEBUG_DDB_ON_USERFAULT.
 1.13 01-Jul-2020  ryo - On some systems with a different cache line size (and DIC,IDC) per CPU, trap "mrs Xt,ctr_el0" instruction
to return the minimum cache line size of the system to userland.
- add CLIDR_EL1 and CTR_EL0 to struct aarch64_sysctl_cpu_id.

On most systems, cache line size is the same for all CPUs, so this mechanism won't be required.
Rather, this is primarily for errata support, which will be committed later.
 1.12 29-Jun-2020  riastradh Draft fpu_kern_enter/leave on aarch64.
 1.11 23-May-2020  ryo Not only the kernel thread, but also the userland PAC keys
(APIA,APIB,APDA,APDB,APGA) are now randomly initialized at exec, and switched
when context switch.
userland programs are able to perform pointer authentication on ARMv8.3+PAC cpu.

reviewd by maxv@, thanks.
 1.10 15-Feb-2020  skrll Various updates and improvements to cpu start up on arm/aarch64

- start sharing more code around the AP startup messaging.
- call arm_cpu_topology_set early so that ci_core_id is available for
drivers, e.g. bcm2835_intr.c
- both arm and aarch64 now have
- a static cpu_info_store array
- the same arm_cpu_{hatched,mbox}
 1.9 18-Dec-2019  riastradh branches: 1.9.2;
New function cpu_startup_hook on arm.

Called at end of cpu_startup. Can be defined in, e.g., evbarm to do
additional stuff after cpu_startup. Defined as a weak alias to a
function that does nothing, so optional.

ok jmcneill
 1.8 16-Jul-2019  skrll branches: 1.8.2;
Add vaddr_t initarm(void *);

Missed in previous commit.
 1.7 06-Apr-2019  thorpej Overhaul the API used to fetch and store individual memory cells in
userspace. The old fetch(9) and store(9) APIs (fubyte(), fuword(),
subyte(), suword(), etc.) are retired and replaced with new ufetch(9)
and ustore(9) APIs that can return proper error codes, etc. and are
implemented consistently across all platforms. The interrupt-safe
variants are no longer supported (and several of the existing attempts
at fuswintr(), etc. were buggy and not actually interrupt-safe).

Also augmement the ucas(9) API, making it consistently available on
all plaforms, supporting uniprocessor and multiprocessor systems, even
those that do not have CAS or LL/SC primitives.

Welcome to NetBSD 8.99.37.
 1.6 18-Oct-2018  skrll Provide generic start code that assumes the MMU is off and caches are
disabled as per the linux booting protocol for ARMv6 and ARMv7 boards.
u-boot image type should be changed to 'linux' for correct behaviour.

The new start code builds a minimal "bootstrap" L1PT with cached access
disabled and uses the same table for all processors. AP startup is
performed in less steps and more code is written in C.

The bootstrap tables and stack are placed into an (orphaned) section
"_init_memory" which is given to uvm when it is no longer used.

Various kernels have been converted to use this code and tested. Some
boards were provided by TNF. Thanks!

The GENERIC kernel now boots on boards using the TEGRA, SUNXI and EXYNOS
kernels. The GENERIC kernel will also work on RPI2 using u-boot.

Thanks to martin@ and aymeric@ for testing on parallella and nanosoc
respectively
 1.5 15-Sep-2018  jakllsch make kernel-groveling crash(8) work on aarch64
 1.4 05-Aug-2018  skrll Refactor code to split aarch{32,64} kernel page tables and VM setup. This
will help re-build the kernel page tables on aarch64 with correct section
mappings.
 1.3 19-Jul-2018  christos Implement TRAP_SIGDEBUG for aarch64...
ptraced programs die with:
data_abort_handler, 257: pid 199.1 (a.out): signal 11 (trap 0x82000006) @pc 0, addr 0x0, error=Instruction Abort (EL0)
 1.2 09-Jul-2018  ryo add MULTIPROCESSOR support
 1.1 01-Apr-2018  ryo branches: 1.1.2; 1.1.4;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1.4.2 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.1.4.1 10-Jun-2019  christos Sync with HEAD
 1.1.2.6 20-Oct-2018  pgoyette Sync with head
 1.1.2.5 30-Sep-2018  pgoyette Ssync with HEAD
 1.1.2.4 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.1.2.3 28-Jul-2018  pgoyette Sync with HEAD
 1.1.2.2 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.2.1 01-Apr-2018  pgoyette file machdep.h was added on branch pgoyette-compat on 2018-04-07 04:12:11 +0000
 1.8.2.1 12-Feb-2020  martin Pull up following revision(s) (requested by riastradh in ticket #705):

sys/arch/aarch64/aarch64/aarch64_machdep.c: revision 1.35
sys/stand/efiboot/efifdt.c: revision 1.20
sys/stand/efiboot/efifdt.h: revision 1.7
sys/arch/aarch64/include/machdep.h: revision 1.9
sys/stand/efiboot/efiboot.h: revision 1.11
sys/arch/arm/arm32/arm32_machdep.c: revision 1.129
sys/arch/arm/include/arm32/machdep.h: revision 1.30
sys/stand/efiboot/exec.c: revision 1.12
sys/arch/evbarm/fdt/fdt_machdep.c: revision 1.65
sys/stand/efiboot/version: revision 1.14
sys/stand/efiboot/boot.c: revision 1.19

New function cpu_startup_hook on arm.

Called at end of cpu_startup. Can be defined in, e.g., evbarm to do
additional stuff after cpu_startup. Defined as a weak alias to a
function that does nothing, so optional.
ok jmcneill

Implement rndseed support in efiboot and fdt arm.

The EFI environment variable `rndseed' specifies the path to the
random seed. It is loaded only for fdt platforms at the moment.
Since the rndseed (an rndsave_t object as defined in <sys/rndio.h>)
is 536 bytes long (for hysterical raisins), and to avoid having to
erase parts of the fdt tree, we load it into a physical page whose
address is passed in the fdt tree, rather than passing the content of
the file as an fdt node directly; the kernel then reserves the page
from uvm, and maps it into kva to call rnd_seed.

For now, the only kernel that does use efiboot with fdt is evbarm,
which knows to handle the rndseed. Any new kernels that use efiboot
with fdt must do the same; otherwise uvm may hand out the page with
the secret key on it for a normal page allocation in the kernel --
which should be OK if there are no kernel memory disclosure bugs, but
would lead to worse consequences than simply loading the seed late in
userland with /etc/rc.d/random_seed otherwise.

ok jmcneill
 1.9.2.1 29-Feb-2020  ad Sync with head.
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file math.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.4 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.3 27-Feb-2018  skrll branches: 1.3.2;
Remove c&p comment that doesn't apply
 1.2 15-Feb-2018  kamil Introduce _UC_MACHINE_FP() as a macro

_UC_MACHINE_FP() is a helper macro to extract from mcontext a frame pointer.

Don't rely on this interface as a compiler might strip frame pointer or
optimize it making this interface unreliable.


For hppa assume a small frame context, for larger frames FP might be located
in a different register (4 instead of 3).

For ia64 there is no strict frame pointer, and registers might rotate.
Reuse 79 following:

./gcc/config/ia64/ia64.h:#define HARD_FRAME_POINTER_REGNUM LOC_REG (79)

Once ia64 will mature, this should be revisited.

A macro can encapsulate a real function for extracting Frame Pointer on
more complex CPUs / ABIs.


For the remaining CPUs, reuse standard register as defined in appropriate ABI.

The direct users of this macro are LLVM and GCC with Sanitizers.

Proposed on tech-userlevel@.

Sponsored by <The NetBSD Foundation>
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.22;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.22.3 21-Mar-2018  martin Pull up the following, requested by kamil in ticket #552:

external/gpl3/gcc{.old}/dist/libsanitizer/asan/asan_linux.cc 1.4
sys/arch/aarch64/include/mcontext.h 1.2
sys/arch/alpha/include/mcontext.h 1.9
sys/arch/amd64/include/mcontext.h 1.19
sys/arch/arm/include/mcontext.h 1.19
sys/arch/hppa/include/mcontext.h 1.9
sys/arch/i386/include/mcontext.h 1.14
sys/arch/ia64/include/mcontext.h 1.6
sys/arch/m68k/include/mcontext.h 1.10
sys/arch/mips/include/mcontext.h 1.22
sys/arch/or1k/include/mcontext.h 1.2
sys/arch/powerpc/include/mcontext.h 1.18
sys/arch/riscv/include/mcontext.h 1.5
sys/arch/sh3/include/mcontext.h 1.11
sys/arch/sparc/include/mcontext.h 1.14-1.17
sys/arch/sparc64/include/mcontext.h 1.10
sys/arch/vax/include/mcontext.h 1.9
tests/lib/libc/sys/Makefile 1.50
tests/lib/libc/sys/t_ucontext.c 1.2-1.5
sys/arch/hppa/include/mcontext.h 1.10
sys/arch/ia64/include/mcontext.h 1.7

- Introduce _UC_MACHINE_FP(). _UC_MACHINE_FP() is a helper
macro to extract from mcontext a frame pointer.
- Add new tests in lib/libc/sys/t_ucontext:
* ucontext_sp (testing _UC_MACHINE_SP)
* ucontext_fp (testing _UC_MACHINE_FP)
* ucontext_pc (testing _UC_MACHINE_PC)
* ucontext_intrv (testing _UC_MACHINE_INTRV)

Add a dummy implementation of _UC_MACHINE_INTRV() for ia64.

Implement _UC_MACHINE_INTRV() for hppa.

Make the t_ucontext.c test more portable.

We now have _UC_MACHINE_FP.
 1.1.22.2 26-Feb-2018  snj revert ticket 552, which broke the build
 1.1.22.1 25-Feb-2018  snj Pull up following revision(s) (requested by kamil in ticket #552):
sys/arch/aarch64/include/mcontext.h: 1.2
sys/arch/alpha/include/mcontext.h: 1.9
sys/arch/amd64/include/mcontext.h: 1.19
sys/arch/arm/include/mcontext.h: 1.19
sys/arch/hppa/include/mcontext.h: 1.9
sys/arch/i386/include/mcontext.h: 1.14
sys/arch/ia64/include/mcontext.h: 1.6
sys/arch/m68k/include/mcontext.h: 1.10
sys/arch/mips/include/mcontext.h: 1.22
sys/arch/or1k/include/mcontext.h: 1.2
sys/arch/powerpc/include/mcontext.h: 1.18
sys/arch/riscv/include/mcontext.h: 1.5
sys/arch/sh3/include/mcontext.h: 1.11
sys/arch/sparc/include/mcontext.h: 1.14-1.17
sys/arch/sparc64/include/mcontext.h: 1.10
sys/arch/vax/include/mcontext.h: 1.9
tests/lib/libc/sys/Makefile: 1.50
tests/lib/libc/sys/t_ucontext.c: 1.2
Introduce _UC_MACHINE_FP() as a macro
_UC_MACHINE_FP() is a helper macro to extract from mcontext a frame pointer.
Don't rely on this interface as a compiler might strip frame pointer or
optimize it making this interface unreliable.
For hppa assume a small frame context, for larger frames FP might be located
in a different register (4 instead of 3).
For ia64 there is no strict frame pointer, and registers might rotate.
Reuse 79 following:
./gcc/config/ia64/ia64.h:#define HARD_FRAME_POINTER_REGNUM LOC_REG (79)
Once ia64 will mature, this should be revisited.
A macro can encapsulate a real function for extracting Frame Pointer on
more complex CPUs / ABIs.
For the remaining CPUs, reuse standard register as defined in appropriate ABI.
The direct users of this macro are LLVM and GCC with Sanitizers.
Proposed on tech-userlevel@.
Sponsored by <The NetBSD Foundation>
--
Improve _UC_MACHINE_FP() for SPARC/SPARC64
Introduce a static inline function _uc_machine_fp() that contains improved
caluclation of a frame pointer.
Algorithm:
uptr *stk_ptr;
# if defined (__arch64__)
stk_ptr = (uptr *) (*sp + 2047);
# else
stk_ptr = (uptr *) *sp;
# endif
*bp = stk_ptr[15];
Noted by <mrg>
--
Make _UC_MACHINE_FP() compile again and fix it so that it does not add
the offset twice.
--
fix _UC_MACHINE32_FP() -- use 32 bit pointer value so that [15] is
the right offset. do this by using __greg32_t, which is only in
the sparc64 version, and these are only useful there, so move them.
--
Add new tests in lib/libc/sys/t_ucontext
New tests:
- ucontext_sp
- ucontext_fp
- ucontext_pc
- ucontext_intrv
They test respectively:
- _UC_MACHINE_SP
- _UC_MACHINE_FP
- _UC_MACHINE_PC
- _UC_MACHINE_INTRV
These tests attempt to access and print the values from ucontext, without
interpreting the values.
This is a follow up of the _UC_MACHINE_FP() introduction.
These tests use PRIxREGISTER, and require to be built with -D_KERNTYPES.
Sponsored by <The NetBSD Foundation>
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file mcontext.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.3.2.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.2 12-Aug-2020  skrll Part III of ad's performance improvements for aarch64

- Assembly language stubs for mutex_enter() and mutex_exit().
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file mutex.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.4 02-Jul-2020  rin Add support of ptrace(2) for COMPAT_NETBSD32.

Now, GDB for arm32 is usable for debugging 32bit applications.

OK ryo@
 1.3 24-Nov-2019  rin PR port-arm/54702

Add support for earmv6hf binaries on COMPAT_NETBSD32 for aarch64:

- Emulate ARMv6 instructions with cache operations register (c7), that
are deprecated since ARMv7, and disabled on ARMv8 with LP64 kernel.

- ep_machine_arch (default: earmv7hf) is copied from executables, as we
do for mips64. "uname -p" reports earmv6hf if compiled for earmv6hf;
configure scripts etc can determine the appropriate architecture.

Many thanks to ryo@ for helping me to add support of Thumb-mode,
as well as providing exhaustive test cases:

https://github.com/ryo/mcr_test/

We've confirmed:

- Emulation works in Thumb-mode.
- T32 16-bit length illegal instruction results in SIGILL, even if
it is located nearby a boundary b/w mapped and unmapped pages.
- T32 32-bit instruction results in SIGSEGV if it is located across
a boundary b/w mapped and unmapped pages.

XXX
pullup to netbsd-9
 1.2 12-Oct-2018  ryo branches: 1.2.4;
add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.
 1.1 01-Apr-2018  ryo branches: 1.1.2; 1.1.4;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1.4.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.1.4.1 10-Jun-2019  christos Sync with HEAD
 1.1.2.3 20-Oct-2018  pgoyette Sync with head
 1.1.2.2 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.2.1 01-Apr-2018  pgoyette file netbsd32_machdep.h was added on branch pgoyette-compat on 2018-04-07 04:12:11 +0000
 1.2.4.1 01-Jan-2021  martin Pull up following revision(s) (requested by rin in ticket #1172):

sys/arch/aarch64/aarch64/trap.c: revision 1.30
sys/arch/aarch64/include/ptrace.h: revision 1.10
sys/arch/aarch64/include/netbsd32_machdep.h: revision 1.4 (patch)
sys/arch/aarch64/aarch64/netbsd32_machdep.c: revision 1.14
sys/arch/aarch64/aarch64/netbsd32_machdep.c: revision 1.15

Add support of ptrace(2) for COMPAT_NETBSD32.

Now, GDB for arm32 is usable for debugging 32bit applications.
OK ryo@

For rev 1.14 and before, netbsd32_process_write_regs() returns EINVAL
if non-modifiable bits are set in CPSR.
Instead, mask out non-modifiable bits and make this function success
regardless of value in CPSR. New behavior matches that of arm:
https://nxr.netbsd.org/xref/src/sys/arch/arm/arm/process_machdep.c#187

This fixes lib/libc/sys/t_ptrace_wait*:access_regs6 tests, in which
register contents retrieved by PT_GETREGS are set back by PT_SETREGS.

No new regression is observed in full ATF run.

OK ryo
 1.16 31-May-2021  simonb Include "opt_param.h" (ifdef _KERNEL_OPT) everywhere that MSGBUFSIZE is
referenced since some sources include <machine/param.h>.
 1.15 24-Jan-2021  jmcneill branches: 1.15.4;
Use 32K as the default NFSv3 read and write data sizes on aarch64, matching
i386 and amd64.
 1.14 01-Feb-2020  skrll branches: 1.14.6;
G/C
 1.13 24-Nov-2019  rin branches: 1.13.2;
PR port-arm/54702

Add support for earmv6hf binaries on COMPAT_NETBSD32 for aarch64:

- Emulate ARMv6 instructions with cache operations register (c7), that
are deprecated since ARMv7, and disabled on ARMv8 with LP64 kernel.

- ep_machine_arch (default: earmv7hf) is copied from executables, as we
do for mips64. "uname -p" reports earmv6hf if compiled for earmv6hf;
configure scripts etc can determine the appropriate architecture.

Many thanks to ryo@ for helping me to add support of Thumb-mode,
as well as providing exhaustive test cases:

https://github.com/ryo/mcr_test/

We've confirmed:

- Emulation works in Thumb-mode.
- T32 16-bit length illegal instruction results in SIGILL, even if
it is located nearby a boundary b/w mapped and unmapped pages.
- T32 32-bit instruction results in SIGSEGV if it is located across
a boundary b/w mapped and unmapped pages.

XXX
pullup to netbsd-9
 1.12 19-Oct-2019  jmcneill Increase aarch64 MAXCPUS to 256.
 1.11 19-Jan-2019  skrll branches: 1.11.4;
Increase MSGBUFSIZE
 1.10 07-Jan-2019  jdolecek move DEV_BSIZE, DEV_BSHIFT out of MD param.h, they are same on all ports

also move BLKDEV_IOSIZE, MAXPHYS, but allow override since some ports
have different value (powerpc uses NBPG for BLKDEV_IOSIZE, sun2/sun3
have lower MAXPHYS)
 1.9 04-Jan-2019  rin ALIGNBYTES32 should be (8 - 1), not (4 - 1) for EABI:
https://nxr.netbsd.org/xref/src/sys/arch/arm/include/cdefs.h#56

Now, sshd for earmv7hf works without problems.
Also fix other users of cmsg(3) API hopefully.
 1.8 06-Dec-2018  skrll Expose CACHE_LINE_SIZE (and COHERENCY_UNIT) so that fstat can work
 1.7 18-Nov-2018  skrll Add CPU_THUNDERX which sets COHERENCY_UNIT and CACHE_LINE_SIZE to 128
 1.6 15-Nov-2018  riastradh Respect the __HIDE_DELAY kludge like on other ports.
 1.5 14-Nov-2018  jakllsch Switch to NKMEMPAGES_MAX_UNLIMITED.

This aligns aarch64 with our other modern 64-bit ports. Significantly
improves file caching utilization on aarch64 systems with copious RAM.
 1.4 12-Oct-2018  ryo add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.
 1.3 28-Apr-2018  jmcneill branches: 1.3.2;
Increase default MSGBUFSIZE to match arm32 defaults
 1.2 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.7 26-Jan-2019  pgoyette Sync with HEAD
 1.1.28.6 18-Jan-2019  pgoyette Synch with HEAD
 1.1.28.5 26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.1.28.4 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.1.28.3 20-Oct-2018  pgoyette Sync with head
 1.1.28.2 02-May-2018  pgoyette Synch with HEAD
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file param.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.3.2.3 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.3.2.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.3.2.1 10-Jun-2019  christos Sync with HEAD
 1.11.4.1 23-Oct-2019  martin Pull up following revision(s) (requested by jmcneill in ticket #359):

sys/arch/aarch64/aarch64/locore.S: revision 1.42
sys/arch/aarch64/aarch64/locore.S: revision 1.43
sys/arch/aarch64/aarch64/locore.S: revision 1.44
sys/arch/arm/fdt/cpu_fdt.c: revision 1.28
sys/arch/aarch64/include/cpu.h: revision 1.14
sys/arch/aarch64/include/param.h: revision 1.12
sys/arch/arm/arm32/cpu.c: revision 1.133
sys/arch/arm/arm32/cpu.c: revision 1.134
sys/arch/arm/include/cpu.h: revision 1.101
sys/arch/arm/acpi/cpu_acpi.c: revision 1.7
sys/arch/aarch64/aarch64/cpu.c: revision 1.23
sys/arch/aarch64/aarch64/cpu.c: revision 1.24
sys/arch/aarch64/aarch64/cpu.c: revision 1.25

Increase aarch64 MAXCPUS to 256.

-

Invalidate dcache before polling AP hatched status

-

Avoid overlap between BP and last AP stack. AP stacks are now in order of
increasing address order.

Spotted by and idea from mlelstv.

-

Use separate cacheline aligned arrays for mbox and hatched as before.

-

cpu_hatched_p only for MULTIPROCESSOR
 1.13.2.1 29-Feb-2020  ad Sync with head.
 1.14.6.1 03-Apr-2021  thorpej Sync with HEAD.
 1.15.4.1 17-Jun-2021  thorpej Sync w/ HEAD.
 1.2 27-Dec-2018  mrg make savecore for arm64 basically work.

- move MD lwp "md_ktf" member into struct pcb. the pcb is used by
the gdb "bsd-kvm" target code to find the stack of each thread
and needs to be available in a well known location.
- implement aarch64_nbsd_supply_pcb() in GDB. makes basic gdb work
on a crash dump.
- remove '#if L_MD_KTF + 8 == L_MD_CPACR' conditional code, as there
is no more L_MD_KTF.

with this gdb has minimal working functionality with "target kvm",
and crash can at least "ps" on a crash dump.

ok skrll.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28; 1.1.30;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.30.1 10-Jun-2019  christos Sync with HEAD
 1.1.28.1 18-Jan-2019  pgoyette Synch with HEAD
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file pcb.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.59 02-Aug-2023  skrll No need to define cpu_{,set}_tlb_info here - just use the
sys/uvm/pmap/pmap_tlb.h versions.
 1.58 20-Apr-2023  skrll Provide a shared pmap_devmap implementation and convert all pmap_devmap
arrays to use DEVMAP_ENTRY{,_END}
 1.57 03-Nov-2022  skrll Provide MI PMAP support on AARCH64
 1.56 29-Oct-2022  skrll KNF + remove unnecessary brackets
 1.55 23-Oct-2022  skrll KNF.
 1.54 23-Oct-2022  skrll Line continuation alignment whitespace. NFC.
 1.53 15-Oct-2022  jmcneill Use "non-posted" instead of "strongly ordered" to describe nGnRnE mappings

Rename the following defines:
- _ARM_BUS_SPACE_MAP_STRONGLY_ORDERED to BUS_SPACE_MAP_NONPOSTED
- PMAP_DEV_SO to PMAP_DEV_NP
- LX_BLKPAG_ATTR_DEVICE_MEM_SO to LX_BLKPAG_ATTR_DEVICE_MEM_NP
Rename the following option:
- AARCH64_DEVICE_MEM_STRONGLY_ORDERED to AARCH64_DEVICE_MEM_NONPOSTED
 1.52 02-Apr-2022  skrll Update to support EFI runtime outside the kernel virtual address space
by creating an EFI RT pmap that can be activated / deactivated when
required.

Adds support for EFI RT to ARM_MMU_EXTENDED (ASID) 32-bit Arm machines.

On Arm64 the usage of pmapboot_enter is reduced and the mappings are
created much later in the boot process -- now in cpu_startup_hook.
Backward compatiblity for KVA mapped RT from old bootaa64.efi is
maintained.

Adding support to other platforms should be easier as a result.
 1.51 15-Jan-2022  skrll Remove unnecessary brackets
 1.50 14-Jan-2022  skrll Restore the previous pmap_remove_all behaviour as the new method meant
the n1sdp couldn't complete a build.

No noticeable change in kernel build performance.
 1.49 10-Oct-2021  skrll Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>
 1.48 19-May-2021  skrll Make even more pmap agnostic
 1.47 30-Apr-2021  skrll branches: 1.47.2;
Make the ddb for pmap / pte information pmap agnostic
 1.46 20-Mar-2021  skrll branches: 1.46.2;
Make pmapboot_enter panic if anything goes wrong and any mappings overlap
rather than only doing it in locore.S
 1.45 31-Jan-2021  skrll branches: 1.45.2;
Improve a comment
 1.44 31-Jan-2021  ryo implement pmap_remove_all().

The size of struct pv_entry has increased, but speed of kernel build has improved by about 1%
exec and exit should have been improved.
 1.43 19-Sep-2020  skrll branches: 1.43.2;
Make __md_palloc pmap agnostic (think sys/uvm/pmap)
 1.42 12-Aug-2020  skrll Part IV of ad's performance improvements for aarch64

- Implement pmap_growkernel(), and update kernel pmap's stats with atomics.

- Then, pmap_kenter_pa() and pmap_kremove() no longer need to allocate
memory nor take pm_lock, because they only modify L3 PTEs.

- Then, pm_lock and pp_lock can be adaptive mutexes at IPL_NONE which are
cheaper than spin mutexes.

- Take the pmap's lock in pmap_extract() if not the kernel's pmap, otherwise
pmap_extract() might see inconsistent state.
 1.41 16-Jul-2020  skrll pmapboot_enter simplication
- bootpage_alloc in asm becomes pmapboot_pagealloc in C
- PMAPBOOT_ENTER_NOBLOCK is removed as it's not used
- PMAPBOOT_ENTER_NOOVERWRITE is removed as it's now always on
- physpage_allocator argument is removed as it's always
pmapboot_pagealloc
- Support for EARLYCONS without CONSADDR is removed so that the identity
map for CONSADDR is always known.

For the assembly files:
2 files changed, 40 insertions(+), 89 deletions(-)

LGTM ryo
 1.40 14-Jun-2020  ad - Fix a lock order reversal in pmap_page_protect().

- Make sure pmap is always locked when updating stats; atomics no longer
needed to do that.

- Remove unneeded traversal of pv list in pmap_enter_pv().

- Shrink struct vm_page from 136 to 128 bytes (cache line sized) and struct
pv_entry from 48 to 32 bytes (power of 2 sized).

- Embed a pv_entry in each vm_page. This means PV entries don't need to
be allocated for private anonymous memory / COW pages / most UBC mappings.
Dynamic PV entries are then used only for stuff like shared libraries and
shared memory.

Proposed on port-arm@.
 1.39 14-May-2020  skrll Use MUTEX_NODEBUG for PV locks as is commonly done. OK ryo.
 1.38 13-May-2020  ryo - move aarch64 addressspace macros from pmap.h to cpufunc.h
- rename ptr_strip_pac() to aarch64_strip_pac()
 1.37 08-Apr-2020  ryo use PMAP_PAGE_INIT() to initialize mutex in pmap_page.

VM_MDPAGE_INIT() in pmap_free_pdp() had initialized pp_flags,
so it unintentionally cleared PMAP_PAGE_FLAGS_PV_TRACKED.
use PMAP_PAGE_INIT to avoid using PMAP_PAGE_FLAGS_PV_TRACKED.

pointed out by tnn@, thanks
 1.36 29-Feb-2020  ryo add helper function aarch64_addresspace() and aarch64_untag_address() to check address space, and eliminate address tag
 1.35 29-Feb-2020  ryo replace KSEG pages mapping code with generic function pmapboot_enter_range()
 1.34 10-Feb-2020  ryo use LIST(3) instead of TAILQ(3) to save one word in struct vm_page and struct pmap.

pointed out by riastradh@. thanks
 1.33 03-Feb-2020  ryo add support pmap_pv(9)

Patch originally from jmcneill@. thanks
 1.32 03-Feb-2020  ryo separate struct vm_page_md into vm_page_md and pmap_page
for preparation pmap_pv(9)
 1.31 26-Jan-2020  skrll Typo in comment
 1.30 06-Jan-2020  skrll branches: 1.30.2;
Fix DEVMAP build losage by reducing diffs between arm and aarch64

*sigh*
 1.29 30-Dec-2019  skrll Drop DEVMAP_{TRUNK_ADDR,ROUND_SIZE} to 4KB pages now that pmap_map_chunk
allows this.
 1.28 28-Dec-2019  jmcneill Do not use Early Write Acknowledge for PCIe I/O and config space.
 1.27 27-Dec-2019  jmcneill Enable early write acknowledge for device memory mappings.
 1.26 29-Oct-2019  maya Define PMAP_NEED_PROCWR, providing strategically placed i-cache
synchronization where just-changed memory is about to be executed.

Fixes SIGILLs seen when running Mono 6 on QEMU Cortex-A57.

ok ryo
 1.25 12-Aug-2019  skrll Use PMAP_DEV in DEVMAP_ENTRY rather than pmap_map_chunk. It's clearer and
means pmap_map_chunk can be made to map other memory types.
 1.24 08-Apr-2019  ryo branches: 1.24.4;
- free empty page tables pages if reach a certain usage.
- need to lock at removing an old pg (_pmap_remove_pv) in _pmap_enter()
 1.23 19-Mar-2019  ryo - add ddb command "machine ttbr" to dump MMU tables.
- tidy up descriptions, usages and messages.
 1.22 19-Mar-2019  ryo - free L1-L3 pages that has been emptied by pmap_remove().
- if no memories, pmap_enter will return correctly ENOMEM if PMAP_CANFAIL, or wait until available any memories if !PMAP_CANFAIL.

These changes improves the stability when we use a huge virtual memory spaces with mmap.
 1.21 06-Feb-2019  ryo improve pmap_remove
- don't lock/unlock per page in pmap_remove()
- speedup pte lookup for continuous addresses
- bring out pool_cache_put(&_pmap_pv_pool, pv) from lock/unlock section
 1.20 04-Jan-2019  jdolecek re-apply rev. 1.18, now tested by Jonathan Kollasch and Ryo Shimizu - no
problems observed, and about 2x speedup for cached read

Implement PMAP_DIRECT / pmap_direct_process() in support of experimental
UBC optimization

PR kern/53124
 1.19 21-Nov-2018  jdolecek revert PMAP_DIRECT until tested; requested by mrg@
 1.18 20-Nov-2018  jdolecek Implement PMAP_DIRECT / pmap_direct_process() in support of experimental
UBC optimizations (compile-tested only for now)

PR kern/53124
 1.17 01-Nov-2018  maxv Add kASan support for aarch64. Stack tracking needs more investigation
and will come in a separate commit.

Reviewed by ryo@ jmcneill@ skrll@.
 1.16 18-Oct-2018  skrll Provide generic start code that assumes the MMU is off and caches are
disabled as per the linux booting protocol for ARMv6 and ARMv7 boards.
u-boot image type should be changed to 'linux' for correct behaviour.

The new start code builds a minimal "bootstrap" L1PT with cached access
disabled and uses the same table for all processors. AP startup is
performed in less steps and more code is written in C.

The bootstrap tables and stack are placed into an (orphaned) section
"_init_memory" which is given to uvm when it is no longer used.

Various kernels have been converted to use this code and tested. Some
boards were provided by TNF. Thanks!

The GENERIC kernel now boots on boards using the TEGRA, SUNXI and EXYNOS
kernels. The GENERIC kernel will also work on RPI2 using u-boot.

Thanks to martin@ and aymeric@ for testing on parallella and nanosoc
respectively
 1.15 13-Oct-2018  ryo - define PMAP_{MAP,UNMAP}_POOLPAGE for performance
- define __HAVE_MM_MD_KERNACC and add mm_md_kernacc()
 1.14 12-Oct-2018  ryo add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.
 1.13 12-Oct-2018  ryo rewrite pmap_pte_lookup() to share similar code.
 1.12 04-Oct-2018  ryo cleanup locore, and changed the way to map memories during boot.
- add functions bootpage_enter() and bootpage_alloc() to adapt various layout
of physical memory map. especially for 64bit physical memory layout.
pmapboot_alloc() allocates pagetable pages from _end[].
- changed to map only the required amount for PA=VA identity mapping
(kernel image, UART device, and FDT blob) with L2_BLOCK(2Mbyte).
- changing page permission for kernel image, and making KSEG mapping are done
at cpu_kernel_vm_init() instead of at locore.
- optimize PTE entries with PTE Contiguous bit. it is enabled on devmap only for now.

reviewed by skrll@, thanks.
 1.11 04-Oct-2018  ryo * define LX_BLKPAG_{OS,ATTR}_* for OS dependent PTE attributes in pmap.h
* cleanup macros
 1.10 15-Sep-2018  jakllsch make kernel-groveling crash(8) work on aarch64
 1.9 10-Sep-2018  maxv Rename _pmap_alloc_pdp -> pmap_alloc_pdp, and make it public.
 1.8 10-Aug-2018  ryo treat kernel-exec attr and user-exec attr separately.
kernel cannot execute userland exec page, and user cannot execute kernel page.
 1.7 06-Aug-2018  ryo set kernel text/rodata readonly by default.
add function db_write_text() for setting ddb breakpoint.
 1.6 27-Jul-2018  ryo changes of pmap.c r1.13 seems to be unstable.
In order to invalidate icache, not to invalidate all icache,
but temporary to make the page writable and invalidate target address only.
 1.5 08-Jun-2018  jmcneill branches: 1.5.2;
Provide bs_mmap implementations for bcm283x based boards.

PR: port-arm/53283
Submitted by: Nick Hudson
 1.4 27-Apr-2018  ryo fix instability behavior of bufcache on aarch64.
* fix to return correct ref/mod when PMAP_WIRED.
* changed to keep wired flags in pte instead of pv_entry, and cleanup.
 1.3 09-Apr-2018  jmcneill Fix encoding of MMAP flags for generic_bs_mmap
 1.2 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.10 18-Jan-2019  pgoyette Synch with HEAD
 1.1.28.9 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.1.28.8 20-Oct-2018  pgoyette Sync with head
 1.1.28.7 30-Sep-2018  pgoyette Ssync with HEAD
 1.1.28.6 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.1.28.5 28-Jul-2018  pgoyette Sync with HEAD
 1.1.28.4 25-Jun-2018  pgoyette Sync with HEAD
 1.1.28.3 02-May-2018  pgoyette Synch with HEAD
 1.1.28.2 16-Apr-2018  pgoyette Sync with HEAD, resolve some conflicts
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file pmap.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.5.2.3 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.5.2.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.5.2.1 10-Jun-2019  christos Sync with HEAD
 1.24.4.2 29-Dec-2019  martin Pull up following revision(s) (requested by jmcneill in ticket #586):

sys/arch/arm/nvidia/tegra_pcie.c: revision 1.27
sys/arch/aarch64/aarch64/pmap.c: revision 1.57
sys/arch/aarch64/aarch64/locore.S: revision 1.48
sys/arch/aarch64/include/armreg.h: revision 1.29
sys/arch/aarch64/aarch64/pmap.c: revision 1.58
sys/arch/aarch64/aarch64/locore.S: revision 1.49
sys/arch/arm/acpi/acpipchb.c: revision 1.14
sys/arch/aarch64/aarch64/genassym.cf: revision 1.16
sys/arch/arm/acpi/acpi_machdep.c: revision 1.13
sys/arch/aarch64/include/pmap.h: revision 1.27
sys/arch/aarch64/aarch64/genassym.cf: revision 1.17
sys/arch/aarch64/include/pmap.h: revision 1.28
sys/arch/arm/fdt/pcihost_fdtvar.h: revision 1.3
sys/arch/arm/include/bus_defs.h: revision 1.14
sys/arch/aarch64/aarch64/bus_space.c: revision 1.9
sys/arch/arm/fdt/pcihost_fdt.c: revision 1.12
sys/arch/aarch64/conf/files.aarch64: revision 1.15
sys/arch/aarch64/conf/files.aarch64: revision 1.16
sys/arch/arm/rockchip/rk3399_pcie.c: revision 1.9

Enable early write acknowledge for device memory mappings.

Do not use Early Write Acknowledge for PCIe I/O and config space.
 1.24.4.1 04-Nov-2019  martin Pull up following revision(s) (requested by maya in ticket #393):

sys/arch/aarch64/include/pmap.h: revision 1.26
sys/arch/aarch64/aarch64/pmap.c: revision 1.48

Define PMAP_NEED_PROCWR, providing strategically placed i-cache
synchronization where just-changed memory is about to be executed.

Fixes SIGILLs seen when running Mono 6 on QEMU Cortex-A57.

ok ryo
 1.30.2.1 29-Feb-2020  ad Sync with head.
 1.43.2.1 03-Apr-2021  thorpej Sync with HEAD.
 1.45.2.1 03-Apr-2021  thorpej Sync with HEAD.
 1.46.2.2 17-Jun-2021  thorpej Sync w/ HEAD.
 1.46.2.1 13-May-2021  thorpej Sync with HEAD.
 1.47.2.1 31-May-2021  cjep sync with head
 1.10 09-Oct-2025  skrll There is no meed to dsb(ishst) after every pte_pde_cas - the necessary
dsb will be performed later when a pte is added,
 1.9 26-Jul-2025  martin Allow building w/o options EFI_RUNTIME
 1.8 26-Jul-2023  skrll branches: 1.8.6;
blank line audit
 1.7 26-Jul-2023  skrll G/C pmap_md_kernel_*
 1.6 26-Jul-2023  skrll Reduce #ifdefs
 1.5 26-Jul-2023  skrll Wrap long lines in a comment block.
 1.4 26-Jul-2023  skrll spaces to tabs.
 1.3 20-Apr-2023  skrll Provide a shared pmap_devmap implementation and convert all pmap_devmap
arrays to use DEVMAP_ENTRY{,_END}
 1.2 21-Dec-2022  skrll Rename pmap_md_pdetab_destroy to pmap_md_pdetab_fini to match
pmap_md_pdetab_init.

Call pmap_md_pdetab_fini from pmap_segtab_destroy.
 1.1 03-Nov-2022  skrll Provide MI PMAP support on AARCH64
 1.8.6.1 02-Aug-2025  perseant Sync with HEAD
 1.2 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file pmc.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.8 12-Aug-2020  skrll Part II of ad's aarch64 performance improvements (cpu_switch.S bugs are
all mine)

- Use tpidr_el1 to hold curlwp and not curcpu, because curlwp is accessed
much more often by MI code. It also makes curlwp preemption safe and
allows aarch64_curlwp() to be a const function (curcpu must be volatile).

- Make ASTs operate per-LWP rather than per-CPU, otherwise sometimes LWPs
can see spurious ASTs (which doesn't cause a problem, it just means some
time may be wasted).

- Use plain stores to set/clear ASTs. Make sure ASTs are always set on the
same CPU as the target LWP, and delivered via IPI if posted from a remote
CPU so that they are resolved quickly.

- Add some cache line padding to struct cpu_info, to match x86.

- Add a memory barrier in a couple of places where ci_curlwp is set. This
is needed whenever an LWP that is resuming on the CPU could hold an
adaptive mutex. The barrier needs to drain the CPU's store buffer, so
that the update to ci_curlwp becomes globally visible before the LWP can
resume and call mutex_exit(). By my reading of the ARM docs it looks like
the instruction I used will do the right thing, but I'm not 100% sure.
 1.7 23-May-2020  ryo Not only the kernel thread, but also the userland PAC keys
(APIA,APIB,APDA,APDB,APGA) are now randomly initialized at exec, and switched
when context switch.
userland programs are able to perform pointer authentication on ARMv8.3+PAC cpu.

reviewd by maxv@, thanks.
 1.6 12-Apr-2020  maxv Add support for Pointer Authentication (PAC).

We use the "pac-ret" option, to sign the return instruction pointer on
function entry, and authenticate it on function exit. This acts as a
mitigation against ROP.

The authentication uses a per-lwp (secret) I-A key stored in the 128bit
APIAKey register and part of the lwp context. During lwp creation, the
kernel generates a random key, and during context switches, it installs
the key of the target lwp on the CPU.

Userland cannot read the APIAKey register directly. However, it can sign
its pointers with it, because the register is architecturally shared
between userland and the kernel. Although part of the CPU design, it is
a bit of an undesired behavior, because it allows to forge valid kernel
pointers from userland. To avoid that, we don't share the key with
userland, and rather switch it in EL0<->EL1 transitions. This means that
when userland executes, a different key is loaded in APIAKey than the one
the kernel uses. For now the userland key is a fixed 128bit zero value.

The DDB stack unwinder is changed to strip the authentication code from
the pointers in lr.

Two problems are known:

* Currently the idlelwps' keys are not really secret. This is because
the RNG is not yet available when we spawn these lwps. Not overly
important, but would be nice to fix with UEFI RNG.
* The key switching in EL0<->EL1 transitions is not the most optimized
code on the planet. Instead of checking aarch64_pac_enabled, it would
be better to hot-patch the code at boot time, but there currently is
no hot-patch support on aarch64.

Tested on Qemu.
 1.5 24-Nov-2019  rin branches: 1.5.6;
part of PR port-arm/54702

Having md_march32 unconditionally in struct mdproc, in order to
make libkvm happy.

XXX
pullup to netbsd-9
 1.4 24-Nov-2019  rin PR port-arm/54702

Add support for earmv6hf binaries on COMPAT_NETBSD32 for aarch64:

- Emulate ARMv6 instructions with cache operations register (c7), that
are deprecated since ARMv7, and disabled on ARMv8 with LP64 kernel.

- ep_machine_arch (default: earmv7hf) is copied from executables, as we
do for mips64. "uname -p" reports earmv6hf if compiled for earmv6hf;
configure scripts etc can determine the appropriate architecture.

Many thanks to ryo@ for helping me to add support of Thumb-mode,
as well as providing exhaustive test cases:

https://github.com/ryo/mcr_test/

We've confirmed:

- Emulation works in Thumb-mode.
- T32 16-bit length illegal instruction results in SIGILL, even if
it is located nearby a boundary b/w mapped and unmapped pages.
- T32 32-bit instruction results in SIGSEGV if it is located across
a boundary b/w mapped and unmapped pages.

XXX
pullup to netbsd-9
 1.3 27-Dec-2018  mrg make savecore for arm64 basically work.

- move MD lwp "md_ktf" member into struct pcb. the pcb is used by
the gdb "bsd-kvm" target code to find the stack of each thread
and needs to be available in a well known location.
- implement aarch64_nbsd_supply_pcb() in GDB. makes basic gdb work
on a crash dump.
- remove '#if L_MD_KTF + 8 == L_MD_CPACR' conditional code, as there
is no more L_MD_KTF.

with this gdb has minimal working functionality with "target kvm",
and crash can at least "ps" on a crash dump.

ok skrll.
 1.2 01-Apr-2018  ryo branches: 1.2.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.2 18-Jan-2019  pgoyette Synch with HEAD
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file proc.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2.2.3 21-Apr-2020  martin Sync with HEAD
 1.2.2.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.2.2.1 10-Jun-2019  christos Sync with HEAD
 1.5.6.1 20-Apr-2020  bouyer Sync with HEAD
 1.4 10-Feb-2021  ryo Oh...the name of the mcount call was different between gcc and llvm.
gcc calls it as "_mconut", llvm calls as "__mcount".

Change the main name of mcount to "mcount()", and created "_mcount" and "__mcount" entries
to work regardless of which compiler the object was created with.
 1.3 10-Feb-2021  ryo add support kernel profiling on aarch64

- add MCOUNT_ENTER, MCOUNT_EXIT macro
- __mcount() function should be aligned
- add "-fno-optimize-sibling-calls" option when PROF. for accurate profiling, it is better to suppress the tail call.
 1.2 23-Apr-2020  jakllsch branches: 1.2.2;
Fix userland gprof profiling on aarch64.

Adjusts _PROF_PROLOGUE to match OpenBSD; reworks our MCOUNT to retrieve
frompc placed on stack by the prologue, and to streamline sp manipulation
when preserving argument registers.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.40;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.40.1 25-Apr-2020  bouyer Sync with bouyer-xenpvh-base2 (HEAD)
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file profile.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2.2.1 03-Apr-2021  thorpej Sync with HEAD.
 1.2 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file psl.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.14 19-Aug-2022  ryo Fixed a bug that pte's __BIT(63,48) could be set when accessing addresses above 0x0001000000000000 in /dev/mem with mmap().
 1.13 10-Oct-2021  skrll Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>
 1.12 29-Feb-2020  ryo Fix pmap to work correctly with tagged addresses

- when fault, untag from address before passing to uvm/pmap functions
- pmap_extract() checks more strictly and consider the address tag
 1.11 31-Jan-2020  maxv BTI definitions.
 1.10 11-Sep-2019  skrll branches: 1.10.2;
Define PRIxPTE
 1.9 11-Sep-2019  skrll Move the TCR and TTBR defines into armreg.h where they below. NFCI.
 1.8 11-Sep-2019  jmcneill - Fix TCR_TG0 field definitions to match Armv8 ARM
- Rename TCR_IPS_64TB to TCR_IPS_16TB, add TCR_IPS_4PB
- Whitespace fixes
 1.7 15-Aug-2019  skrll Indent the field value defines. NFCI.
 1.6 13-Aug-2019  skrll Add DBM
 1.5 04-Oct-2018  ryo * define LX_BLKPAG_{OS,ATTR}_* for OS dependent PTE attributes in pmap.h
* cleanup macros
 1.4 17-Jul-2018  ryo fix build with aarch64 gcc/gas
 1.3 01-Apr-2018  ryo branches: 1.3.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.2 16-Jan-2017  maya branches: 1.2.12;
Correct definitions for TCR.

Values from ARM Cortex A-53 MPCore Processor Technical Reference Manual
4.3.48. Translation Control Register, EL1
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.6; 1.1.10; 1.1.14;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.14.1 21-Apr-2017  bouyer Sync with HEAD
 1.1.10.1 20-Mar-2017  pgoyette Sync with HEAD
 1.1.6.1 05-Feb-2017  skrll Sync with HEAD
 1.1.4.3 03-Dec-2017  jdolecek update from HEAD
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file pte.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2.12.3 20-Oct-2018  pgoyette Sync with head
 1.2.12.2 28-Jul-2018  pgoyette Sync with HEAD
 1.2.12.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.3.2.2 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.3.2.1 10-Jun-2019  christos Sync with HEAD
 1.10.2.1 29-Feb-2020  ad Sync with head.
 1.12 07-Sep-2020  ryo Oops. revert my previous commit. AArch64 instructions are always LE.
 1.11 06-Sep-2020  ryo need swap for aarch64be
 1.10 02-Jul-2020  rin Add support of ptrace(2) for COMPAT_NETBSD32.

Now, GDB for arm32 is usable for debugging 32bit applications.

OK ryo@
 1.9 18-Jun-2019  kamil branches: 1.9.2;
Introduce PTRACE_REG_FP() a helper macro to retrieve the frame pointer

The macro is dummy for ia64 (the FP register is unknown and can change
freely) and sparc/sparc64 (not stored in struct reg).
 1.8 13-Dec-2018  ryo add support PT_STEP
 1.7 21-Jul-2018  ryo don't depend endian.
 1.6 20-Jul-2018  christos flip the byte order
 1.5 12-Apr-2017  kamil branches: 1.5.10; 1.5.12;
Add new macro PTRACE_BREAKPOINT_ASM in <sys/ptrace.h> MD part

This macro ships with a MD-specific assembly instruction triggering
a software breakpoint.

Missing instruction for powerpc targets.

This code is used in ATF tests (lib/libc/sys/t_ptrace_wait).

Original patch by Nick Hudson, thanks!
 1.4 25-Sep-2015  christos branches: 1.4.2; 1.4.4;
For processors that have memory breakpoints, add macros for them to help
libproc
 1.3 15-Sep-2015  christos Provide access to pc/sp/syscall-return registers like we have for mcontext
 1.2 11-Aug-2014  matt branches: 1.2.2; 1.2.4;
#include <arm/ptrace.h> instead of <arm/asm.h>
(opps)
 1.1 10-Aug-2014  matt Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.2.4.3 28-Aug-2017  skrll Sync with HEAD
 1.2.4.2 27-Dec-2015  skrll Sync with HEAD (as of 26th Dec)
 1.2.4.1 22-Sep-2015  skrll Sync with HEAD
 1.2.2.3 03-Dec-2017  jdolecek update from HEAD
 1.2.2.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.2.1 11-Aug-2014  tls file ptrace.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.4.4.1 21-Apr-2017  bouyer Sync with HEAD
 1.4.2.1 26-Apr-2017  pgoyette Sync with HEAD
 1.5.12.2 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.5.12.1 10-Jun-2019  christos Sync with HEAD
 1.5.10.2 26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.5.10.1 28-Jul-2018  pgoyette Sync with HEAD
 1.9.2.1 01-Jan-2021  martin Pull up following revision(s) (requested by rin in ticket #1172):

sys/arch/aarch64/aarch64/trap.c: revision 1.30
sys/arch/aarch64/include/ptrace.h: revision 1.10
sys/arch/aarch64/include/netbsd32_machdep.h: revision 1.4 (patch)
sys/arch/aarch64/aarch64/netbsd32_machdep.c: revision 1.14
sys/arch/aarch64/aarch64/netbsd32_machdep.c: revision 1.15

Add support of ptrace(2) for COMPAT_NETBSD32.

Now, GDB for arm32 is usable for debugging 32bit applications.
OK ryo@

For rev 1.14 and before, netbsd32_process_write_regs() returns EINVAL
if non-modifiable bits are set in CPSR.
Instead, mask out non-modifiable bits and make this function success
regardless of value in CPSR. New behavior matches that of arm:
https://nxr.netbsd.org/xref/src/sys/arch/arm/arm/process_machdep.c#187

This fixes lib/libc/sys/t_ptrace_wait*:access_regs6 tests, in which
register contents retrieved by PT_GETREGS are set back by PT_SETREGS.

No new regression is observed in full ATF run.

OK ryo
 1.3 17-Jul-2018  kamil Use __uint128_t conditionally in aarch64 reg.h

Check whether __uint128_t is available checking __SIZEOF_INT128__ in
preprocessor.
Move __aligned attribute to the whole structure.

No functional change for current NetBSD/aarch64 users of GCC and Clang.

This change allows to use the aarch64 target with rumpkernel on Linux
aarch64 hosts, in a toolchain configuration with 128-bit variables.

OK from <martin> and <christos>
 1.2 01-Apr-2018  ryo branches: 1.2.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.2 28-Jul-2018  pgoyette Sync with HEAD
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file reg.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2.2.1 10-Jun-2019  christos Sync with HEAD
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file rwlock.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.2 10-May-2020  skrll Don't futz with tpidr_el0 in {set,long}jmp as it breaks TLS as seen in
qemu
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.34;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.34.1 13-May-2020  martin Pull up following revision(s) (requested by skrll in ticket #901):

sys/arch/aarch64/include/setjmp.h: revision 1.2
lib/libc/arch/aarch64/genassym.cf: revision 1.2
lib/libc/arch/aarch64/gen/setjmp.S: revision 1.3
lib/libc/arch/aarch64/gen/_setjmp.S: revision 1.4

Don't futz with tpidr_el0 in {set,long}jmp as it breaks TLS as seen in
qemu
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file setjmp.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.4 05-Nov-2021  thorpej Normally, to support COMPAT_NETBSD32 we need to define
__HAVE_STRUCT_SIGCONTEXT in order to support the old
"sigcontext" style of handlers for 32-bit binaries.
However, we only support 32-bit EABI binaries on AArch64,
and by happy accident (due to a libc bug introduced in
2006), 32-bit NetBSD EABI binaries never used "sigcontext"
style handlers. So, we don't need to carry any of this
baggage forward.

This addresses the AArch64 case of PR kern/56487.
 1.3 27-Oct-2021  thorpej - In sendsig() and sigaction1(), don't hard-code signal trampoline
versions. Instead, use the version constants from <sys/signal.h>
and automatically (and correctly) handle cases where multiple versions
of a particular trampoline flavor exist. Conditionalize support
for sigcontext trampolines on __HAVE_STRUCT_SIGCONTEXT.
- aarch64 and amd64 don't use sigcontext natively, but do need to
support it for 32-bit compatibility; define __HAVE_STRUCT_SIGCONTEXT
conditionally on _KERNEL.
 1.2 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file signal.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.5 05-May-2024  riastradh aarch64/sljit_machdep.h: Make this work in compat32 context.

Should fix clang build of compat32 eabi libsljit:

dependall ===> compat/arm/eabi/../../../lib/../external/bsd/sljit/lib
In file included from /home/source/ab/HEAD-llvm/src/sys/external/bsd/sljit/dist/sljit_src/sljitLir.c:1678:
/home/source/ab/HEAD-llvm/src/sys/external/bsd/sljit/dist/sljit_src/sljitNativeARM_64.c:142:54: error: shift count >= width of type [-Werror,-Wshift-count-overflow]
FAIL_IF(push_inst(compiler, MOVK | RD(dst) | (((imm >> 32) & 0xffff) << 5) | (2 << 21)));
^ ~~
 1.4 02-Apr-2024  riastradh bsd.own.mk: Enable MKLSJIT on aarch64.

Make sure there's only one copy of the conditional, in bsd.own.mk;
just make sys/modules/Makefile conditional on MKSLJIT so we don't
have to keep these in sync.

As a workaround for PR 58106, tweak the conditional definition of
SLJIT_CACHE_FLUSH to use cpu_icache_sync_range only in _HARDKERNEL,
and use __builtin___clear_cache in userland and in rump kernels.

PR 58103: bpfjit.kmod is not built on aarch64
 1.3 11-Dec-2020  skrll branches: 1.3.18;
s:aarch64/cpufunc.h:arm/cpufunc.h:

a baby step in the grand arm header unification challenge
 1.2 02-Dec-2018  alnsn branches: 1.2.4; 1.2.6; 1.2.14;
Switch to __builtin___clear_cache() in userspace.

aarch64_sync_icache() doesn't exist because there no libarm equivalent
on aarch64.
 1.1 26-Aug-2018  rjs branches: 1.1.2;
Add SLJIT to aarch64.
 1.1.2.3 26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.1.2.2 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.1.2.1 26-Aug-2018  pgoyette file sljit_machdep.h was added on branch pgoyette-compat on 2018-09-06 06:55:23 +0000
 1.2.14.1 14-Dec-2020  thorpej Sync w/ HEAD.
 1.2.6.1 18-Apr-2024  martin Pull up following revision(s) (requested by riastradh in ticket #655):

sys/modules/Makefile: revision 1.285
share/mk/bsd.own.mk: revision 1.1365
share/mk/bsd.own.mk: revision 1.1366
sys/arch/aarch64/include/sljit_machdep.h: revision 1.4
sys/external/bsd/sljit/dist/sljit_src/sljitNativeARM_64.c: revision 1.5
(all via patch)

sljit: Pacify -Wsign-compare.

If these sizes are negative, we're probably in trouble anyway, so
assert nonnegative here.
Needed to resolve PR 58103.

bsd.own.mk: Enable MKLSJIT on aarch64.

Make sure there's only one copy of the conditional, in bsd.own.mk;
just make sys/modules/Makefile conditional on MKSLJIT so we don't
have to keep these in sync.

As a workaround for PR 58106, tweak the conditional definition of
SLJIT_CACHE_FLUSH to use cpu_icache_sync_range only in _HARDKERNEL,
and use __builtin___clear_cache in userland and in rump kernels.

PR 58103: bpfjit.kmod is not built on aarch64
bsd.own.mk: No need for MKSLJIT to be set differently from others.
- Use ?=, not =, so mk.conf setting wins.
- Write out per-architecture tabular settings, not a conditional.
- Add comments for the architectures that look like they should have
sljit but don't. (XXX Missing comments about powerpc and mips --
not sure why, is this because modules don't yet work on those
architectures, or what?)

Tidying for PR 58103: bpfjit.kmod is not built on aarch64.
 1.2.4.2 10-Jun-2019  christos Sync with HEAD
 1.2.4.1 02-Dec-2018  christos file sljit_machdep.h was added on branch phil-wifi on 2019-06-10 22:05:43 +0000
 1.3.18.2 11-May-2024  martin Additionally pull up the following, to fix 32bit compat compilation after
ticket #655, requested by riastradh:

sys/arch/aarch64/include/sljit_machdep.h 1.5

aarch64/sljit_machdep.h: Make this work in compat32 context.
 1.3.18.1 18-Apr-2024  martin Pull up following revision(s) (requested by riastradh in ticket #655):

sys/modules/Makefile: revision 1.285
share/mk/bsd.own.mk: revision 1.1365
share/mk/bsd.own.mk: revision 1.1366
sys/arch/aarch64/include/sljit_machdep.h: revision 1.4
sys/external/bsd/sljit/dist/sljit_src/sljitNativeARM_64.c: revision 1.5

sljit: Pacify -Wsign-compare.

If these sizes are negative, we're probably in trouble anyway, so
assert nonnegative here.
Needed to resolve PR 58103.

bsd.own.mk: Enable MKLSJIT on aarch64.

Make sure there's only one copy of the conditional, in bsd.own.mk;
just make sys/modules/Makefile conditional on MKSLJIT so we don't
have to keep these in sync.

As a workaround for PR 58106, tweak the conditional definition of
SLJIT_CACHE_FLUSH to use cpu_icache_sync_range only in _HARDKERNEL,
and use __builtin___clear_cache in userland and in rump kernels.

PR 58103: bpfjit.kmod is not built on aarch64
bsd.own.mk: No need for MKSLJIT to be set differently from others.
- Use ?=, not =, so mk.conf setting wins.
- Write out per-architecture tabular settings, not a conditional.
- Add comments for the architectures that look like they should have
sljit but don't. (XXX Missing comments about powerpc and mips --
not sure why, is this because modules don't yet work on those
architectures, or what?)

Tidying for PR 58103: bpfjit.kmod is not built on aarch64.
 1.2 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file sysarch.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.3 10-May-2020  skrll branches: 1.3.2;
Provide a trap.h (currently empty)
 1.2 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.28;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.28.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file trap.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.3.2.2 13-May-2020  martin Pull up following revision(s) (requested by skrll in ticket #900):

sys/arch/aarch64/include/Makefile: revision 1.4
sys/arch/aarch64/include/trap.h: revision 1.3
distrib/sets/lists/comp/ad.aarch64: revision 1.40

Provide a trap.h (currently empty)

Update for trap.h
 1.3.2.1 10-May-2020  martin file trap.h was added on branch netbsd-9 on 2020-05-13 12:31:11 +0000
 1.21 03-Nov-2022  skrll Provide MI PMAP support on AARCH64
 1.20 10-Oct-2021  skrll Use sys/uvm/pmap/pmap_tlb.c on Aarch64 in the same way that some Arm, MIPS,
and some PPC kernels do. This removes the limitation of 256 processes on
CPUs with 8bit ASID field, e.g. Apple M1.

Additionally the following changes have been made

- removed a couple of unnecessary aarch64_tlbi_all calls
- removed any invalidation after freeing page tables due to
_pmap_sweep_pdp. This was never necessary afaict.
- all kernel mappings are marked global and userland mapping not-global.

Performance testing hasn't show a significant difference. The data here
is from building a kernel on an lx2k system with nvme.

before
1489.6u 400.4s 2:40.65 1176.5% 228+224k 0+32289io 57pf+0w
1482.6u 403.2s 2:38.49 1189.9% 228+222k 0+32274io 46pf+0w
1485.4u 402.2s 2:37.27 1200.2% 228+222k 0+32275io 12pf+0w

after
1493.9u 404.6s 2:37.50 1205.4% 227+221k 0+32265io 48pf+0w
1485.0u 408.0s 2:38.54 1194.0% 227+222k 0+32272io 36pf+0w
1484.3u 407.0s 2:35.88 1213.3% 228+224k 0+32268io 14pf+0w

>>> stats.ttest_ind([160.65,158.49,157.27], [157.5,158.54,155.88])
Ttest_indResult(statistic=1.1923622711296888, pvalue=0.2990182944606766)
>>>
 1.19 30-Sep-2021  skrll Make tlb_asid_t unsigned int as pmap_tlb.c expects tlb_asid_t to be able to
hold ASID_MAX + 1.
 1.18 24-Mar-2021  simonb s/depreciated/deprecated/g
 1.17 23-Jan-2021  jmcneill branches: 1.17.2;
Add __HAVE_BUS_SPACE_8
 1.16 14-Sep-2020  ryo branches: 1.16.2;
PID_MAX is just an initial value (soft maximum). Don't use it for CTASSERT.
defined __HAVE_CPU_MAXPROC to use function cpu_maxproc().

pointed out by mrg@, thanks.
 1.15 03-Aug-2020  ryo Implement MD ucas(9) (__HAVE_UCAS_FULL)
 1.14 14-Feb-2020  skrll sort __HAVE_* defines. NFCI
 1.13 06-Dec-2019  kamil branches: 1.13.2;
Remove __HAVE_CPU_LWP_SETPRIVATE from aarch64

aarch64 specific cpu_lwp_setprivate() is redundant with its caller
lwp_setprivate() and there are no MD bits.
 1.12 13-Oct-2018  ryo - define PMAP_{MAP,UNMAP}_POOLPAGE for performance
- define __HAVE_MM_MD_KERNACC and add mm_md_kernacc()
 1.11 17-Jul-2018  joerg Be consistent and explicitly size register32_t too.
 1.10 17-Jul-2018  christos match declaration types for registers from reg.h
 1.9 12-Jul-2018  maxv Remove the kernel PMC code. Sent yesterday on tech-kern@.

This change:

* Removes "options PERFCTRS", the associated includes, and the associated
ifdefs. In doing so, it removes several XXXSMPs in the MI code, which is
good.

* Removes the PMC code of ARM XSCALE.

* Removes all the pmc.h files. They were all empty, except for ARM XSCALE.

* Reorders the x86 PMC code not to rely on the legacy pmc.h file. The
definitions are put in sysarch.h.

* Removes the kern/sys_pmc.c file, and along with it, the sys_pmc_control
and sys_pmc_get_info syscalls. They are marked as OBSOL in kern,
netbsd32 and rump.

* Removes the pmc_evid_t and pmc_ctr_t types.

* Removes all the associated man pages. The sets are marked as obsolete.
 1.8 28-Apr-2018  jmcneill branches: 1.8.2;
Define __HAVE_OLD_DISKLABEL for compatibility with the arm32 port.
 1.7 27-Apr-2018  ryo define __HAVE_ATOMIC64_OPS
pointed out by nonaka@, thanks
 1.6 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.5 28-Feb-2016  joerg branches: 1.5.16;
Reorder using register_t to the point where it is defined.
 1.4 23-Jan-2016  christos expose the kernel types for standalone code.
 1.3 23-Jan-2016  christos Hide {p,v}{addr,size}_t and register_t (and a couple more types that
are machine-specific) from userland unless _KERNEL/_KMEMUSER and a
new _KERNTYPES variables is defined. The _KERNTYPES should be fixed
for many subsystems that should not be using it (rump)...
 1.2 27-Aug-2015  pooka Fix PTHREAD_FOO_INITIALIZER for C++ by not using volatile in the relevant
pthread types in C++ builds, attempt 2.

The problem with attempt 1 was making assumptions of what the MD
__cpu_simple_lock_t (declared volatile) looks like. To get a same type
except non-volatile, we change the MD type to __cpu_simple_lock_nv_t
and typedef __cpu_simple_lock_t as a volatile __cpu_simple_lock_nv_t.
IMO, __cpu_simple_lock_t should not be volatile at all, but changing it
now is too risky.

Fixes at least Rumprun w/ gcc 5.1/5.2. Furthermore, the mpd application
(and possibly others) will no longer require NetBSD-specific patches.

Tested: build.sh for i386, Rumprun for x86_64 w/ gcc 5.2.

Based on the patch from Christos in lib/49989.
 1.1 10-Aug-2014  matt branches: 1.1.4; 1.1.6;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.6.2 19-Mar-2016  skrll Sync with HEAD
 1.1.6.1 22-Sep-2015  skrll Sync with HEAD
 1.1.4.3 03-Dec-2017  jdolecek update from HEAD
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file types.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.5.16.4 20-Oct-2018  pgoyette Sync with head
 1.5.16.3 28-Jul-2018  pgoyette Sync with HEAD
 1.5.16.2 02-May-2018  pgoyette Synch with HEAD
 1.5.16.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.8.2.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.8.2.1 10-Jun-2019  christos Sync with HEAD
 1.13.2.1 29-Feb-2020  ad Sync with head.
 1.16.2.1 03-Apr-2021  thorpej Sync with HEAD.
 1.17.2.1 03-Apr-2021  thorpej Sync with HEAD.
 1.1 01-Apr-2018  ryo branches: 1.1.2;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1.2.2 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.2.1 01-Apr-2018  pgoyette file userret.h was added on branch pgoyette-compat on 2018-04-07 04:12:11 +0000
 1.21 30-Jun-2024  jmcneill aarch64: Bump VM_PHYSSEG_MAX to match DRAM_BANKS / FDT_MEMORY_RANGES.

On aarch64 there is a single free list, so VM_PHYSSEG_MAX needs to be
the same as FDT_MEMORY_RANGES (which needs to be the same as DRAM_BANKS).
Future cleanup should be done to fold these into a single define.
 1.20 16-Apr-2023  skrll branches: 1.20.6;
Rename VM_KERNEL_IO_ADDRESS to VM_KERNEL_IO_BASE to match RISC-V

It's less letters, matches other similar variables and will help with
sharing code between the two architectures.

NFCI.
 1.19 02-Apr-2022  skrll branches: 1.19.4;
Update to support EFI runtime outside the kernel virtual address space
by creating an EFI RT pmap that can be activated / deactivated when
required.

Adds support for EFI RT to ARM_MMU_EXTENDED (ASID) 32-bit Arm machines.

On Arm64 the usage of pmapboot_enter is reduced and the mappings are
created much later in the boot process -- now in cpu_startup_hook.
Backward compatiblity for KVA mapped RT from old bootaa64.efi is
maintained.

Adding support to other platforms should be easier as a result.
 1.18 21-Mar-2021  skrll Adjust the kernel virtual address space so that KASAN will map the kernel
seperately from managed kernel virtual memory and not map the unused space
between the two.
 1.17 10-Nov-2020  skrll branches: 1.17.2;
AA64 is not MIPS.

Change all KSEG references to directmap
 1.16 06-Oct-2020  christos branches: 1.16.2;
GC unused MAXTSIZ32
 1.15 23-Sep-2020  skrll Readability of a comment
 1.14 19-Sep-2020  skrll Define VM_KERNEL_VM_{BASE,SIZE} for aarch64 and remove an #ifdef in
fdt/platform.h

NFCI
 1.13 16-Sep-2020  skrll G/C AARCH64_KMEMORY_BASE
 1.12 08-Jul-2020  skrll Fix a comment
 1.11 04-Mar-2020  ryo change kernel vm base address to use more than 256GB of memory. (up to 64TB)

also enlarge KSEG(direct map) region from 512GB to 64TB.
KASAN works ok.

Note: -fasan-shadow-offset=
KASAN_SHADOW_START - (CANONICAL_BASE >> 3) =
0xFFFF400000000000 - (0xFFFF000000000000 >> 3) =
0xDFFF600000000000
 1.10 22-Jan-2020  ad Bump UBC defaults on sparc64 & aarch64, which already have a large pager_map.
 1.9 21-Jan-2020  jmcneill Switch aarch64 to use a single freelist.
 1.8 28-Oct-2018  jmcneill branches: 1.8.6;
Document the VA range reserved for EFI runtime services.
 1.7 12-Oct-2018  ryo add initial support of COMPAT_NETBSD32 on AArch64.
arm ELF32 EABI binaries could be execute in AArch32 state on AArch64. A32 THUMB mode is not supported yet.
 1.6 14-Sep-2018  ryo define VM_KERNEL_IO_SIZE for clarity
 1.5 07-Sep-2018  jmcneill Increase VM_PHYSSEG_MAX to 64
 1.4 12-May-2018  jmcneill branches: 1.4.2;
Increase PAGER_MAP_DEFAULT_SIZE to 512MB (from 16MB)
 1.3 01-Apr-2018  ryo Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.2 11-Aug-2014  matt branches: 1.2.2; 1.2.20;
Add some definitions for building RUMP libraries with MKCOMPAT.
 1.1 10-Aug-2014  matt Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.2.20.5 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.2.20.4 20-Oct-2018  pgoyette Sync with head
 1.2.20.3 30-Sep-2018  pgoyette Ssync with HEAD
 1.2.20.2 21-May-2018  pgoyette Sync with HEAD
 1.2.20.1 07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.2.2.3 03-Dec-2017  jdolecek update from HEAD
 1.2.2.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.2.2.1 11-Aug-2014  tls file vmparam.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000
 1.4.2.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.4.2.1 10-Jun-2019  christos Sync with HEAD
 1.8.6.1 25-Jan-2020  ad Sync with head.
 1.16.2.2 03-Apr-2021  thorpej Sync with HEAD.
 1.16.2.1 14-Dec-2020  thorpej Sync w/ HEAD.
 1.17.2.1 03-Apr-2021  thorpej Sync with HEAD.
 1.19.4.1 03-Jul-2024  martin Pull up following revision(s) (requested by jmcneill in ticket #735):

sys/dev/pci/pci_resource.c: revision 1.5
sys/arch/arm/pci/pci_msi_machdep.c: revision 1.10
sys/arch/aarch64/include/vmparam.h: revision 1.21
sys/dev/acpi/acpi_resource.c: revision 1.43

pci_resource: Make unexpected bus numbers in bridges non-fatal.

Firmware bugs happen. Log a warning and continue instead of panicing.
acpi: Ignore producer/consumer bit for fixed memory resources.

The requirement to honour the producer/consumer bit in fixed memory
resource descriptors was dropped at some point in a revision to the ACPI
2.0 specification because too many firmware implementations got it wrong.

aarch64: Bump VM_PHYSSEG_MAX to match DRAM_BANKS / FDT_MEMORY_RANGES.

On aarch64 there is a single free list, so VM_PHYSSEG_MAX needs to be
the same as FDT_MEMORY_RANGES (which needs to be the same as DRAM_BANKS).

Future cleanup should be done to fold these into a single define.

arm: pci: Fix ITS ID lookup for MSIs.
pci_get_frameid expects a BDF requestor ID as input, not a Device ID.

Fixes MSI/MSI-X support on Ampere Altra systems.
 1.20.6.1 01-Jul-2024  perseant Sync with HEAD.
 1.1 10-Aug-2014  matt branches: 1.1.4;
Preliminary files for AARCH64 (64-bit ARM) support.
Enough for a distribution build.
 1.1.4.2 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.4.1 10-Aug-2014  tls file wchar_limits.h was added on branch tls-maxphys on 2014-08-20 00:02:39 +0000

RSS XML Feed