Convert remaining armv7 kernels to generic start and remove a bunch of
code including a9_mpsubr.S.

Thanks to Lwazi Dube for testing PANDABOARD and BEAGLEBONE.

Centralise defparam CONSADDR, CONSPEED, CONMODE and CONADDR into
opt_console.h and adjust.

branches: 1.57.2;
on evbarm/RPI,RPI2, VERBOSE_INIT_ARM had broken.

XPUTC() of evbarm/rpi/rpi*_start.S uses bcm283[567]_platform_early_putchar() and it requires stack.
fixed to allocate stack when starting from rpi*_start.S and a9_mpsubr.S if needed.

to work XPUTC(), need to define VERBOSE_INIT_ARM and EARLYCONS option.

keep stack pointer even if chainging CPU mode.

branches: 1.55.2;
Use r0 in the tlbiall instruction to avoid confusion.

From Ramakrishna Rao Desetti

FDTise RapberryPI support. Thanks for jmcneill for a lot of help with this.

The kernel image that the RPI firmware boots is now netbsd.img in the
kernel build directory.

XXX fdtbus_get_reg needs reworking

Add #define CPU_CORTEDVIRT to reduce copied complex ifdef.
Shrink HYP test

Ensure CNTVOFF is 0 before dropping out of Hyp mode
CVS ----------------------------------------------------------------------

Typo

Clear TEX Remap Enable in SCTLR. U-Boot with CONFIG_ARMV8_SWITCH_TO_EL1
sets it since it is RES1 when TTBCR.EAE=1, but we are still using the
short descriptor format and don't currently support TEX remap.

ok skrll@

Replace assembler flags with .arch* annotation.

Make fp 0 on entry to idle_stack to ensure we mark that we're top of
stack

branches: 1.47.8;
Don't touch ACTLR on CORTEX A9, if ACTLR.SMP=1. like Linux.
Tested on Gumstix DuoVero(OMAP 4430).

branches: 1.46.2;
Deal with kernel builds where virtualisation isn't available

Check for hypervisor mode in cortex_init and exit if the cpu started
there.

Needed by latest RPI firmware.

G/C TEGRAK1_PMAP_WORKAROUND.

Using XN (eXecute Never) properly means speculative reads from devices
aren't done. Pretty sure this was the cause of the wedges.

Setting actlr.bit15=1 (Force in order issue in the branch execution unit)
makes my jetson tk1 stable. Apply this workaround until we figure out
what the real problem is.

Use TTBR_[UM]PATTR in a9_mpsubr.S as well as cpufunc_asm_armv7

Prompted by matt@

Add CALL macro to hide ugliness of calling when KERNEL_BASE_VOFFSET is non-0

Call armv7_dcache_l1inv_all in mpstart
Make sure ACTRL<31> is set on A15 with 3+ cores.

A15 change for > 2 cores.

Add a missing defined(CORTEX_A15)

Make sure TLB is invalidated and ACTLR.SMP is set on ARM A15. ACTLR.SMP
enables the processor to receive instruction cache, BTB and TLB main-
tenance operations from other processors

On secondary cores, invalidate the caches to make them clean.

Remove unused label

ARM spells the System Control Register SCTLR

isb between TTBCR write and TLBIALL

Restore "#if !defined(CPU_CORTEXA5)" accidentally removed in previous
commit -- hi skrll@

Use character constants instead of ascii values for readability

Spin up the non-boot CPUs, but don't allow cpu_boot_secondary_processors
to see them for now.

RPI2 nows works well with only the boot cpu.

skip a TLBIALL on Cortex-A5 that stops my odroid-c1 from booting, ok matt

According to the Cortex-A5 TRM, the CBAR register is not implemented and
always reads as 0x00000000. Add ARM_CBAR option to set this in kernel
config.

make sure we set ACTLR.SMP=1 for CPU_CORTEXA5 in !MP case, ok matt@

Fix CORTEXA17 support

Add Cortex-A17 support

branches: 1.24.2;
Always set CORTEXA9_AUXCTL_SMP on the CortexA9 as well as A7

Always set SMP on the A7 since it's needed for ldrex/strex

Enable L2 prefetch on A9 if MP

Don't set CORTEXA9_AUXCTL_FW on A7. It's undefined.

Misc. cleanup while I'm here.

fix to invalidate all way of own processor

Make sure to read SCTRL no matter way CPU we are on.

branches: 1.18.2;
Fix wrong instruction; mrc => mcr

whitespace cleanup

Simplify calls into .text by using KERNEL_BASE_VOFFSET.

Fix a typo - it's "#define". Use defined(foo) rather than just foo in
preprocessor conditional.

ok matt@

branches: 1.14.2;
Improve MP startup code. We now use a two stage startup, after creating
the initial L1PT and turning on the MMU/caches, we spinup the secondary CPUs
waiting for them to get the same state as the boot processor. Once the
real L1PT is initialized and used, the secondary CPUs are kicked so they can
use it (and the initial L1PT is discarded). Finally, wait until NetBSD
kicks the secondary CPUs then load the stack from the idlelwp and then hatch
the cpu and then jump to idle_loop.

Rework PIC method to be simplier. Change be more cortex neutral.

branches: 1.12.2;
Deal with writing bswapped com registers at COM_MULT register widths.

dsb is preferred for post-armv7 CPUs.

Deal with ARM32_DISABLE_ALIGNMENT_FAULTS

Remove brain fart with TTBCR

Turn on CPU_CONTROL_EX_BEND for big endian in exceptions,
not CPU_CONTROL_BEND_ENABLE.

Make sure CPU_CONTROL_BEND_ENABLE is set on BE systems.

branches: 1.6.2;
Set caching bits on the TTBR for ARMv7
Make sure TTCR is 0

Make this a little more generic.
Make sure to turn on branch prediction.

Make these compile with gcc4.1 and binutils 2.16

branches: 1.3.2; 1.3.4; 1.3.6;
Force TTBCR to 0

Cleanup and bring forwards from bcm53xx_start.S
Use more symbolic names ...

Add Cortex-A9 support including the ARM Generic Interrupt Controller
and the A9 Global Timer / Watchdog.

Handle speed change events

branches: 1.2.2; 1.2.10;
Various fixes / changes

- don't use prescaler
- improve AB_DEBUG output
- fix a9ptmr_delay to work with a decrementing counter!

Thanks to jmcneill@ for proving I'm an idiot

Add a driver for the A5/A9 Private timer.

While here FDTise the Watchdog driver.

Handle speed change events

branches: 1.1.2; 1.1.10;
Add a driver for the A5/A9 Private timer.

While here FDTise the Watchdog driver.

arm: Use device_set_private for various drivers.

comparision -> comparison

branches: 1.20.10;
Trailing whitespace

Switch the DE0 Nano SoC to the GENERIC kernel.

Enable the global timer at attach time, it ensures that delay() works.

Remove comment that the peripherals clock should be half of the cpu clock.
NFC

branches: 1.16.2;
Use mpcaa_off1 parameter for mapping subregion.

Rename ARM A9 Global Timer driver name to support fdt.

- Rename a9tmr to arma9tmr.
- Add a9tmr_fdt.c based gtmr_fdt.c.

branches: 1.14.16;
KNF

- fix sc_ev_missing_ticks over-counting.
- don't use 64bit division, because it has expensive cost on gcc/arm
whether it is a constant or not.

'delta' is usually taken a value around sc_autoinc depending on timing
of read. therefore 'delta / sc->sc_autoinc' would be count too much.

print "A5" instead of "A9" at attach time if running on a Cortex-A5

match on Cortex-A5

fix typo

detach and re-attach timecounter when updating freq, and reinit timer on each cpu

add a helper to update a9tmr frequency

branches: 1.7.4; 1.7.6;
Mark interrupt as MPSAFE.

branches: 1.6.2; 1.6.6;
Pass the offset from CBAR/PERIPHBASE in mpcore_attach_args.
Modify the list of devices to include the offset(s) from PERIPHBASE.

Don't attach a9tmr if the CPU supports the generic timer.

Make sure we enable the timer since CFE doesn't enable it by default.

branches: 1.3.2; 1.3.4;
Don't bother dealing with "extra" ticks, just count them.

Set dv_private, use a9tmr_private.

branches: 1.1.2;
Add Cortex-A9 support including the ARM Generic Interrupt Controller
and the A9 Global Timer / Watchdog.

branches: 1.1.4; 1.1.6; 1.1.16;
Add Cortex-A9 support including the ARM Generic Interrupt Controller
and the A9 Global Timer / Watchdog.

Trailing whitespace

Rename a register define to make it more obvious it's watchdog only

branches: 1.1.4; 1.1.6; 1.1.16; 1.1.46;
Add Cortex-A9 support including the ARM Generic Interrupt Controller
and the A9 Global Timer / Watchdog.

Add a driver for the A5/A9 Private timer.

While here FDTise the Watchdog driver.

branches: 1.6.2; 1.6.6;
Use mpcaa_off1 parameter for mapping subregion.

Rename ARM A9 Global Timer driver name to support fdt.

- Rename a9tmr to arma9tmr.
- Add a9tmr_fdt.c based gtmr_fdt.c.

branches: 1.4.16;
add a helper to update a9tmr frequency

branches: 1.3.6; 1.3.10; 1.3.12;
Pass the offset from CBAR/PERIPHBASE in mpcore_attach_args.
Modify the list of devices to include the offset(s) from PERIPHBASE.

branches: 1.2.2; 1.2.4;
Don't bother dealing with "extra" ticks, just count them.

branches: 1.1.2;
Add Cortex-A9 support including the ARM Generic Interrupt Controller
and the A9 Global Timer / Watchdog.

Add a driver for the A5/A9 Private timer.

While here FDTise the Watchdog driver.

spaces to tab

branches: 1.8.2;
Fix loop to calculate prescaler divisor.

branches: 1.7.4;
Trailing whitespace

Set attached to true when attach completes.

Reported by Ramakrishna Rao Desetti on port-arm

Register sysmon_wdog.

branches: 1.4.2;
print "A5" instead of "A9" at attach time if running on a Cortex-A5

match on Cortex-A5

branches: 1.2.2; 1.2.4;
write correct register.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.16; 1.1.18;
Add Cortex-A9 support including the ARM Generic Interrupt Controller
and the A9 Global Timer / Watchdog.

Add missing RCSIDs.
Fix typo s/btyes/bytes/ in comment.

branches: 1.19.12;
Merge thorpej-cfargs2.

branches: 1.18.8;
Merge thorpej-cfargs branch:

Simplify and make extensible the config_search() / config_found() /
config_attach() interfaces: rather than having different variants for
which arguments you want pass along, just have a single call that
takes a variadic list of tag-value arguments.

Adjust all call sites:
- Simplify wherever possible; don't pass along arguments that aren't
actually needed.
- Don't be explicit about what interface attribute is attaching if
the device only has one. (More simplification.)
- Add a config_probe() function to be used in indirect configuiration
situations, making is visibly easier to see when indirect config is
in play, and allowing for future change in semantics. (As of now,
this is just a wrapper around config_match(), but that is an
implementation detail.)

Remove unnecessary or redundant interface attributes where they're not
needed.

There are currently 5 "cfargs" defined:
- CFARG_SUBMATCH (submatch function for direct config)
- CFARG_SEARCH (search function for indirect config)
- CFARG_IATTR (interface attribte)
- CFARG_LOCATORS (locators array)
- CFARG_DEVHANDLE (devhandle_t - wraps OFW, ACPI, etc. handles)

...and a sentinel value CFARG_EOL.

Add some extra sanity checking to ensure that interface attributes
aren't ambiguous.

Use CFARG_DEVHANDLE in MI FDT, OFW, and ACPI code, and macppc and shark
ports to associate those device handles with device_t instance. This
will trickle trough to more places over time (need back-end for pre-OFW
Sun OBP; any others?).

branches: 1.17.2;
Fix build

branches: 1.16.2;
Collapse all CPU_CORTEXA<n> options into CPU_CORTEX and do runtime
detection instead of ifdefs where required.

Cortex A5 also uses PPI 27 for timer.

ODROID-C1 boots again.

Sprinkle #include "opt_cputypes.h"

branches: 1.13.2;
Rename ARM A9 Global Timer driver name to support fdt.

- Rename a9tmr to arma9tmr.
- Add a9tmr_fdt.c based gtmr_fdt.c.

branches: 1.12.2;
Set attached to true when attach completes.

Reported by Ramakrishna Rao Desetti on port-arm

branches: 1.11.6;
Recognize Cortex-A57 FPU, GIC, and Generic Timer.

need to include <sys/lwp.h>

Make this compile where gtmr isn't used.

Initial RPI2 support - it doesn't work yet. The generic timer gets messed
up somehow.

This commit changes the KVA layout of the RPI.

allow arml2cc to be used on Cortex-A5 if the "offset" property is specified

match on Cortex-A5

Let the "cbar" device property override the cbar value, to work around broken bootloaders

branches: 1.4.6; 1.4.10; 1.4.12;
Pass the offset from CBAR/PERIPHBASE in mpcore_attach_args.
Modify the list of devices to include the offset(s) from PERIPHBASE.

Add generic timer support (untested)

branches: 1.2.2; 1.2.4; 1.2.6;
Add driver to attach ARM PL210 L2 Cache Controller
arml2cc0 at armperiph0: ARM PL310 L2 r3p2 Cache Controller
arml2cc0: 256KB/32B 16-way L2 Unified cache

Add Cortex-A9 support including the ARM Generic Interrupt Controller
and the A9 Global Timer / Watchdog.

Removed unused file

branches: 1.1.2; 1.1.12;
something temporary that will go away once odroid xu4 works

cpu_in_cksum_asm_neon.S: Nix trailing whitespace.

No functional change intended.

Add missing RCSIDs.
Fix typo s/btyes/bytes/ in comment.

branches: 1.4.10;
Trailing whitespace

branches: 1.3.2; 1.3.6; 1.3.14;
Rework considerably. Use alternating sets of registers.
(Still not faster than normal ARM code).

Tighten up cpu_in_cksum_neon_v4hdr by 3 instructions.
Swap the doubles on a partial qword load on BE platforms.

Add preliminary version of a NEON based in_cksum routine.

Add missing RCSIDs.
Fix typo s/btyes/bytes/ in comment.

branches: 1.1.2; 1.1.6; 1.1.14; 1.1.80;
Add preliminary version of a NEON based in_cksum routine.

Remove GIC_SPLFUNCS.

Make gic_splfuncs optional and disable it by default until it has had
more testing.

Use custom spl funcs for GIC and avoid unnecessary pmr register accesses
in splx.

Collapse all CPU_CORTEXA<n> options into CPU_CORTEX and do runtime
detection instead of ifdefs where required.

Add support for Amazon's Graviton MSI controller.

Graviton has a GICv3 with a modified GICv2m (!) for MSIs. Instead of
sending messages to a fixed address with the SPI as data, the Graviton's
GICv2m uses a different address for each vector with "don't care" as data.

Add a driver for the A5/A9 Private timer.

While here FDTise the Watchdog driver.

branches: 1.11.4;
Support building kernels with ACPI and no PCI.

Add support for GICv3 Locality-specific Periphal Interrupts (LPIs) and the
Interrupt Translation Service (ITS).

Add support for PCI MSI using ARM GICv2m.

Add GICv3 support.

branches: 1.7.2;
Rename ARM A9 Global Timer driver name to support fdt.

- Rename a9tmr to arma9tmr.
- Add a9tmr_fdt.c based gtmr_fdt.c.

branches: 1.6.8;
Move children of armperiph to the "mpcorebus" interface attribute and have
armperiph declare support for it.

branches: 1.5.6;
Add needs-flag to arml2cc

branches: 1.4.2; 1.4.6;
Add generic timer support (untested)

Add preliminary version of a NEON based in_cksum routine.

branches: 1.2.2; 1.2.4; 1.2.6;
Add driver to attach ARM PL210 L2 Cache Controller
arml2cc0 at armperiph0: ARM PL310 L2 r3p2 Cache Controller
arml2cc0: 256KB/32B 16-way L2 Unified cache

Add Cortex-A9 support including the ARM Generic Interrupt Controller
and the A9 Global Timer / Watchdog.

arm/gic: Check l_blcnt, not l_biglocks.

l_biglocks is a now-defunct temporary storage used only when
sleeping; l_blcnt is the number of kernel locks held by the lwp when
not sleeping.

Should fix arm builds.

build fix: remove includes of opt_gic.h

Remove GIC_SPLFUNCS.

pic: Update ci_cpl in pic_set_priority callback.

Not all ICs need interrupts disabled to update the priority. DAIF accesses
are not cheap, so push the update of ci_cpl from pic_set_priority to the
IC's pic_set_priority callback, and let the IC driver determine whether
or not it needs interrupts disabled.

arm: Use device_set_private for various drivers.

arm: Remove #ifdef DIAGNOSTIC now wrong after KASSERT change.

Objects in question aren't volatile here so access is flushable.

Fix some conditionals to match gicv3 and add some comments to describe
what's going on.

Fixes PR port-evbarm/56420

If an SGI or PPI is established after interrupts are enabled, make sure
we unblock the source on _all_ CPUs and not just the CPU that is
establishing the interrupt.

Make gic_splfuncs optional and disable it by default until it has had
more testing.

Use custom spl funcs for GIC and avoid unnecessary pmr register accesses
in splx.

Only target the boot cpu for real with SPI interrupts. I tried to do
this back in 2014, but somehow I missed a spot.

This is a quick-and-dirty fix for the USB stack which expects transfer
completions to be in-order. If interrupts happen across the CPUs then
this isn't guaranteed (yet).

kern/55243 panic at usb_transfer_complete() on raspberry pi 4

branches: 1.46.2;
If we are committing a deferred splhigh() to hardware, no need to continue.

Apply PMR optimizations from gicv3

Avoid an extra daif read when dispatching interrupts by using
ENABLE_INTERRUPT() / DISABLE_INTERRUPT() instead of cpsie() / cpsid() macros.

Provide and use a sev() macro for the sev instruction.

While here use the correct barrier to ensure completion of memory accesses
before a couple of the sev() calls.

branches: 1.42.2;
G/C arm/atomic.h

Remove CPU ID test in armgic_match. Perfectly normal to have GIC in
non-Cortex cores.

Avoid undefined behaviour. Detected by KUBSAN.

Fix "left shift of 255 by 24 places cannot be represented in type 'int'"
warning from UBSan.

branches: 1.38.4; 1.38.10;
Use intr_establish_xname

armgic_ipi_send: use GIC interface number, not CPU index, when setting
the target(s) for an IPI.

Update sc_mptargets atomically, as PEs will be started up in parallel w/o locking here

Support pic_set_affinity and pic_get_affinity

branches: 1.34.2;
Cover all pic_maxsources lines for armgic_cpu_init_priorities() and
armgic_cpu_update_priorities().

Previously only the first 32 lines were covered, which is significantly
less than the 1000-some interrupt lines possible.

Only relevant to MULTIPROCESSOR configurations.

Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)

branches: 1.32.2;
PR# port-evbarm/49468: Cortex GIC assertion triggered on Allwinner A80 SoC

The priority level is changed by writing to GICC_PMR with interrupts
disabled. However, interrupts are enabled/disabled downstream of the GICC
at the CPU. When raising priority level, there is a window between the time
that interrupts are disabled and the GICC_PMR register is written. If an
interrupt occurs at a previously allowed priority before GICC_PMR is
changed, the CPU will receive the signal when interrupts are re-enabled.
At this time, GICC_PMR is now the new priority level, so reads of
GICC_IAR will report a spurious IRQ.

Move the "old_ipl != IPL_HIGH" test until after we have confirmed that
there is at least one pending IRQ.

KNF

ARM Trusted Firmware reserves SGIs 8-15 for secure use. Even without ATF,
U-Boot on some platforms may use SGIs in this range for the PSCI
implementation.

Change ARMGIC_IPI_BASE to 0 from (16 - NIPI) and add a compile-time assert
to ensure that we don't end up with a conflict.

Revert the KERNHIST for now it causes problems

If we see GICC_IAR_IRQ_SSPURIOUS we should also stop looking for irqs

Wrap long line

Print the GICC_IIDR when AB_DEBUG

Add 'armgichist' KERNHIST for (future) debugging.

Don't assume that CPU index = GIC CPU interface number. We can determine
the current CPU interface number by reading from the read-only
GICD_ITARGETSR0 through GICD_ITARGETSR7 registers.

This gets interrupts working on Exynos 5422, where the boot processor has
GIC CPU interface #4.

Clean out some #if 0'ed or //'ed code

Initialise all the SGI/PPI priorities for all CPUs to mask the interrupts

branches: 1.21.2;
Use an FDT-based ARM_INTR_IMPL for Tegra.

Adjust some of IPLs of various IPIs.

Add separate IPI routines for IPI_AST and IPI_KPREEMPT.

Add a pic_cpus to the softc which specifies which cpus the pic can send
IPIs to. For GIC, initialize pic_cpus to kcpuset_running since it can handle
all the cpus.

All SGIs are MPSAFE so establish them as such.

Trailing whitespace

G/C armgic_last_priority

in armgic_establish_irq, make sure to write the new value to GICD_ICFGRn when setting irq type (IST_LEVEL/IST_EDGE)

sc_gic_lines is the total number of valid lines but pic_sources[] is sparse; when initializing mpsafe targets make sure to go all the way to the end (sc_pic.pic_maxsources) instead

branches: 1.12.2;
Only target boot cpu for SPIs atm. Fix a couple of bugs in the now
disabled code.

Sprinkle #include "opt_multiprocessor.h"

branches: 1.10.2;
Implement MI IPI interface with cross-call support.

When dealing with the PMR register, only use non-secure priority values.

Move aprint to print sooner.

branches: 1.7.2;
Various MP changes.

Don't byte swap the data, assume the bus_space will do it.

branches: 1.5.2;
armgic_priority_to_ipl is only used in #if 0'd code, so apply that as
well.

branches: 1.4.2;
Pass the offset from CBAR/PERIPHBASE in mpcore_attach_args.
Modify the list of devices to include the offset(s) from PERIPHBASE.

branches: 1.3.2; 1.3.4;
Rename kcpuset_copybits() to kcpuset_export_u32() and thus be more specific
about the interface.

Verify the source isn't a dummy source.

branches: 1.1.2;
Add Cortex-A9 support including the ARM Generic Interrupt Controller
and the A9 Global Timer / Watchdog.

Use an FDT-based ARM_INTR_IMPL for Tegra.

branches: 1.2.4;
add helper macros.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.16; 1.1.18;
Add Cortex-A9 support including the ARM Generic Interrupt Controller
and the A9 Global Timer / Watchdog.

Add GICD_CTRL_EnableGrp1S definition.

branches: 1.11.10;
Fix definition of GICD_IROUTER; the input to this macro is the INTID,
which can be a value between 32 and 1019.

Handle ThunderX errata

Update GICD_CTLR reg bit definitions to reflect the layout of the register
when either in non-secure state or for a system that only supports a single
state.

Add support for GICv3 Locality-specific Periphal Interrupts (LPIs) and the
Interrupt Translation Service (ITS).

Update for GICv3

branches: 1.6.4; 1.6.6;
Naming consistency for (as yet) unused register

Correct a comment

Add an define for spurious interrupt number 1022

branches: 1.3.4;
When dealing with the PMR register, only use non-secure priority values.

branches: 1.2.2;
Various MP changes.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.16;
Add Cortex-A9 support including the ARM Generic Interrupt Controller
and the A9 Global Timer / Watchdog.

Implement gic_splraise and the gic_splx fast path in asm (armv8).

Add missing insn barrier

Make _splraise/_spllower/splx functions available to modules again.

gic_splx: performance optimizations

Avoid any kind of register access (DAIF, PMR, etc), barriers, and atomic
operations in the common case where no interrupt fires between spl being
raised and lowered.

This introduces a per-CPU return address (ci_splx_restart) used by the
vector handler to restart a sequence in splx that compares the new ipl
with the per-CPU hardware priority state stored in ci_hwpl.

Use custom spl funcs for GIC and avoid unnecessary pmr register accesses
in splx.

Use custom spl funcs for GIC and avoid unnecessary pmr register accesses
in splx.

Reload tpidr_el1 into x3 and re-test newipl vs cpl on restart.

Add __HAVE_PREEMPTION support to gic_splfuncs asm funcs.

"looks right to me" - thorpej

Implement gic_splraise and the gic_splx fast path in asm (armv8).

Remove an unnecessary if statement in gic_v2m_msi_alloc when finding a
'count' that fits the available.

Preserve ST Lower and Upper fields when clearing Mask bit in the MSI-X
vector control register.

branches: 1.9.2;
Do not store a pointer to the passed in struct pci_attach_args

Use PCI_MSIX_"TBL"BIR_MASK instead of PCI_MSIX_"PBA"BIR_MASK for MSI-X table.
This is not a real bug because both macros have the same value.

Add support for Amazon's Graviton MSI controller.

Graviton has a GICv3 with a modified GICv2m (!) for MSIs. Instead of
sending messages to a fixed address with the SPI as data, the Graviton's
GICv2m uses a different address for each vector with "don't care" as data.

branches: 1.6.2;
- Disable MSI/MSI-X when making changes
- MSI: Write the vector count to the Multi Message Enable (MME) field
- MSI: Set DATA to the first LPI number, not the last

branches: 1.5.4;
fix a paste-o in a panic message

branches: 1.4.2;
Add intr_establish_xname support to arm and expose it to intrctl

Add MSI-X support.

Fail gracefully when an attempt to allocate MSI vectors is made on a device
without MSI capabilities.

Add support for PCI MSI using ARM GICv2m.

Do not store a pointer to the passed in struct pci_attach_args

Add support for Amazon's Graviton MSI controller.

Graviton has a GICv3 with a modified GICv2m (!) for MSIs. Instead of
sending messages to a fixed address with the SPI as data, the Graviton's
GICv2m uses a different address for each vector with "don't care" as data.

branches: 1.1.2; 1.1.6; 1.1.8;
Add support for PCI MSI using ARM GICv2m.

gicv3: Some more left-shift v.s. int fixes

Suggested by msaitoh@, thanks!

gicv3: Appease KUBSAN, otherwise NFC

branches: 1.54.4; 1.54.12;
build fix: remove includes of opt_gic.h

gicv3: Treat all IPLs below IPL_VM as IPL_NONE for hwpl mappings

Remove GIC_SPLFUNCS.

pic: Update ci_cpl in pic_set_priority callback.

Not all ICs need interrupts disabled to update the priority. DAIF accesses
are not cheap, so push the update of ci_cpl from pic_set_priority to the
IC's pic_set_priority callback, and let the IC driver determine whether
or not it needs interrupts disabled.

arm/cortex: Use container_of, not bespoke offsetof arithmetic.

Trailing whitespace

If an SGI or PPI is established after interrupts are enabled, make sure
we unblock the source on _all_ CPUs and not just the CPU that is
establishing the interrupt.

GIC: Support SPI affinity routing before secondary CPUs have hatched.

Instead of reading MPIDR from CPU init to determine a routing target, trust
system configuration data (Devicetree or ACPI) to properly fill in ci_cpuid.
This allows for SPI affinity routing to work before secondary CPUs are
hatched; previously, one of two things would happen:

a) If 1ofN distribution is supported, the SPI could be distributed to any
participating PE.
b) If 1ofN distribution is not supported, the SPI would not be forwarded
and would stay pending.

Make gic_splfuncs optional and disable it by default until it has had
more testing.

Use custom spl funcs for GIC and avoid unnecessary pmr register accesses
in splx.

Disable 1ofN distribution of SPIs by default. This is a workaround for an
issue in the USB stack -- signaling transfer complete on multiple PEs can
cause transfer completions to be processed out of order.

branches: 1.43.2;
If we are committing a deferred splhigh() to hardware, no need to continue.

Keep current hardware priority value in struct cpu_info and use it instead
of reading icc_pmr_el1 in gicv3_set_priority.

fix build without MULTIPROCESSOR

Optimize hardware priority updates.

In gicv3_set_priority, read the current priority mask and only update it
if we are lowering the priority. The hardware priority filter is raised
only after taking an interrupt. This makes splfoo/splx without an interrupt
in between considerably cheaper as PMR writes are relatively expensive
compared to reads.

While here, avoid an extra daif read when dispatching interrupts by using
ENABLE_INTERRUPT() / DISABLE_INTERRUPT() instead of cpsie() / cpsid() macros.

Remove MAXCPUS dependency.

When lpiconf flushing is required, make sure to flush to PoC and not PoU.
Spotted by nick.

Fix spelling in comment.

gicv3: Only use 1 of N SPI distribution when the feature is available.

A GICv3+ implementation is not guaranteed to support 1 of N SPI
distribution. Support for this feature is indicated in GICD_TYPER.No1N.

When No1N=1, route all interrupts to the primary PE by default and only
allow a single CPU target when updating affinity.

Improve detection of NS vs S views of priorities.

For PMR, write a 0 to bit7 and see if it sticks. This is only possible from
NS EL1 if we have a non-secure view of ICC_PMR_EL1.

For int priorities (GICD/GICR interfaces and LPIs), assume that the
GICD_CTLR.DS bit is telling us the truth.

RK3399 is special here when using the vendor bootloader, so keep the
auto-detection from the previous commit but limit the scope to only run
on RK3399 SOCs.

Fix interrupt priorities on N1 SDP.

The GICv3 architecture specification is not clear on the NS view of
priority registers, and there doesn't seem to be any consistency in how
these are implemented in both real and emulated environments.

The previous fix for this issue was meant to detect what we thought at the
time was a bug on the Rockchip RK3399. At that time the theory was somehow
EL1 has a secure view of the hardware, and this is causing us to have the
wrong view of IPRIORITYRn based on IHI0069F section 4.8.6 "Software
accesses of interrupt priority". But it turns out that this is not the
full picture. While I was able to confirm that yes, we do have secure
access to the GIC on RK3399 from EL1, the view of IPRIORITYRn differs
depending on whether you are using the Rockchip TF-A as included with
https://github.com/ayufan-rock64/linux-u-boot (shifted view), or mainline
TF-A from pkgsrc (unshifted view).

So to detect this quirk, we need three things: A method to detect if we
have S access to GIC registers, a method to see how many PMR bits are
implemented, and a method to see how many IPRIORITYRn bits are implemented.

To detect S access, we can try to toggle GICD_CTRL.EnableGrp1S. This bit
is either RES0 (security extensions not implemented), RAZ/WI (non-secure
access in two security state systems) or RW (secure access in two security
state systems).

To read the number of PMR and IPRIORITYRn bits supported, we can write all
1s to the register fields and read them back.

For the RK3399 (Rockchip TF-A) quirk, we assume a shifted view of
IPRIORITYRn if we have detected S accesses, and the PMR and IPRIORITYRn
values differ. The S access test is required because some real hardware
implementations (Ampere eMAG) were observed to report different PMR and
IPRIORITYRn masks, but present an unshifted view of IPRIORITYRn.

During testing, I also discovered that QEMU 5.1 requires this shifted view
workaround as well -- as far as I can tell, this is a QEMU bug. We can't
detect it the same way as RK3399 because security is disabled in the
emulated GIC, and the PMR and IPRIORITYRn tests both return 0xff! So now
if the GICv3 driver sees this configuration, it assumes that the shifted
view is required.

Honestly, this feature is so poorly documented that maybe it is better to
give up on HW priorities and preemption and use a single flat model like
Linux and FreeBSD does.

Tested on Arm N1 SDP, ROCKpro64 (RK3399) with Rockchip and pkgsrc TF-A,
Pinebook Pro (RK3399), Lenovo HR330A (Ampere eMAG), QEMU 5.1 (gic-version=3),
AWS EC2 a1.medium (Graviton).

Add a per-CPU event counter that counts every time an interrupt handler is
preempted by a higher priority interrupt.

branches: 1.32.2;
sc_enabled_sgippi can be updated from any CPU, so use atomic_or/atomic_and

Remove unused __HAVE_PIC_FAST_SOFTINTS block. It never would have worked
if enabled..

Add an isb() barrier after ICC_SGI1R_EL1 write to prevent reordering with
subsequent wfi/wfe instructions. Haven't seen this in practice but I would
rather be safe here.

gicv3_set_priority: ICC_PMR_EL1 is self-synchronizing so no need for isb()
here.

gicv3_irq_handler: No need to call gicv3_set_priority if we are already at
the desired ipl.

gicv3_ipi_send: simplify logic in kcp != NULL case given that we know that
the kcpuset will only ever contain one cpu.

Retire arm_[di]sb in favour of the isb() and dsb(sy) macro invocations.

Apply similar fix from gic.c that fixed "left shift of 255 by 24 places
cannot be represented in type 'int'" warnings from UBSan.

branches: 1.24.4;
Make intr affinity work with MSIs again

Since all ITS instances share a common LPI configuration table, used a
shared vmem arena to allocate pic irqs. Idea from FreeBSD.

branches: 1.22.2;
Traiing whitespace

- Use pic_do_pending_ints in intr handler
- Sprinkle isb
- Fix PMR bits detection on eMAG, from OpenBSD

branches: 1.20.2;
Fix size of LPI pending table allocation and enable caching of LPI conf
and pending tables where possible.

Change how we detect secure vs non-secure access.

Write 0xff to ICC_PMR_EL1 and read back how many bits are implemented,
then do the same with a GICD_IPRIORITYR<n> priority value field.

If the values differ, assume we have a shifted view of IPRIORITYR.

Improve priority handling for cases where access is secure, from OpenBSD.

revert rev 1.4:
>Adjust priority mappings, NFCI

it has some unintended change that makes nvme hangy. ok @jmcneill.

Revert "Route all interrupts to the primary PE by default"

Route all interrupts to the primary PE by default

Adjust priority mappings, NFCI

branches: 1.13.4;
Fix LPI pending table size, use correct LPI conf offset in gicv3_lpi_block_irqs, and set bit[7]=1 for G1NS interrupts when writing to the LPI configuration table.

kcpuset_ffs returns the cpu number plus one, so make sure to subtract it

Use intr_establish_xname

Instead of disabling preemption, set the binary point field to the minimum supported value

Update GICD_CTLR reg bit definitions to reflect the layout of the register
when either in non-secure state or for a system that only supports a single
state.

Save a few pages by only allocating LPI pending tables for "ncpu" instead of "MAXCPU" CPUs.

Implement pic_get_affinity/pic_set_affinity for LPIs via ITS

Implement pic_get_affinity/pic_set_affinity for SPIs

Add support for GICv3 Locality-specific Periphal Interrupts (LPIs) and the
Interrupt Translation Service (ITS).

ICC_PMR_EL1 has different encoding than IPRIORITYR. Not 100% sure that this is correct yet, but it works with both RK3399 and QEMU.

Set NS access bit when writing ICC_PMR_EL1

branches: 1.2.2;
IPI and priority fixes. My RK3399 board boots multiuser now.

Add GICv3 support.

Remove MAXCPUS dependency.

gicv3: Only use 1 of N SPI distribution when the feature is available.

A GICv3+ implementation is not guaranteed to support 1 of N SPI
distribution. Support for this feature is indicated in GICD_TYPER.No1N.

When No1N=1, route all interrupts to the primary PE by default and only
allow a single CPU target when updating affinity.

Improve detection of NS vs S views of priorities.

For PMR, write a 0 to bit7 and see if it sticks. This is only possible from
NS EL1 if we have a non-secure view of ICC_PMR_EL1.

For int priorities (GICD/GICR interfaces and LPIs), assume that the
GICD_CTLR.DS bit is telling us the truth.

RK3399 is special here when using the vendor bootloader, so keep the
auto-detection from the previous commit but limit the scope to only run
on RK3399 SOCs.

branches: 1.8.6;
Since all ITS instances share a common LPI configuration table, used a
shared vmem arena to allocate pic irqs. Idea from FreeBSD.

branches: 1.7.4;
Fix size of LPI pending table allocation and enable caching of LPI conf
and pending tables where possible.

Remove unused field

Improve priority handling for cases where access is secure, from OpenBSD.

branches: 1.4.4;
Implement pic_get_affinity/pic_set_affinity for LPIs via ITS

Implement pic_get_affinity/pic_set_affinity for SPIs

Add support for GICv3 Locality-specific Periphal Interrupts (LPIs) and the
Interrupt Translation Service (ITS).

branches: 1.1.2;
Add GICv3 support.

gicv3_its: Fix indirect L2 device table allocation.

gicv3: its: Fix regression on LX2K.

Page size probing and device ID width changes broke LX2K, revert these
parts of the last set of changes as they are not required on X1E.

gicv3: log the ITS command table information

fixup! Restore a line accidentally removed in my previous change

gicv3_its.c: Do-nothing macros must expand to `__nothing', not empty.

No functional change intended -- unless this was accidentally broken
before.

gicv3: ITS: Add support for indirect device tables.

When devbits is too large to fit into the first level of device table,
some hardware supports the concept of indirect tables. Add support for
this and use it when supported.

This is required for MSIs to work properly on Snapdragon X Elite platforms
(otherwise, MAPD command fails and stalls the ITS command queue).

branches: 1.35.2;
gicv3_its: assorted minor improvements

- KASSERT that the command queue is not stalled due to async error
before issuing more commands.
- implement gits_command_int under #if 0 (useful for troubleshooting)
- protect command queue with its_lock consistently
- KASSERT for 32-bit MSI that upper bits are 0
- KASSERT return value of gicv3_its_msi_alloc_lpi
- KASSERT that safe conditions for writing to CBASER are met
- move CWRITER initialization to before CBASER
- fix alignment of tables for the 16KB page size case

Fix non-DIAGNOSTIC builds (hopefully). No idea why this didn't fail to
build for me without DIAGNOSTIC.

Rework Arm (32bit and 64bit) AP startup so that cpu_hatch doesn't sleep.

The AP initialisation code in cpu_init_secondary_processor will read and
initialise the required system registers and state for the BP to attach
and report.

Rework the interrupt handler code for this new sequence. Thankfully,
this removes a bunch of code for bcm2836mp.

The VFP detection handler on <= armv7 relies on the global undefined
handler being in place until the BP attaches vfp. That is, after the
APs have been spun up.

gicv3_its.c has a serialisation issue which is protected against in
the gicv3_its_cpu_init, which is called from cpu_hatch, with a spin
lock. The serialisation issue needs addressing more completely.

Tested on RPI3, Apple M1, QEMU, and lx2k

Fixes PR port-arm/56264:
diagnostic assertion "l->l_stat == LSONPROC" failed on RPI3

Remove MAXCPUS dependency.

trailing whitespace

Simplify gits_command by getting rid of the _BYTE_ORDER test and just
always use the loop + htole64 path.

Preserve ST Lower and Upper fields when clearing Mask bit in the MSI-X
vector control register.

branches: 1.28.2;
need to swap on big endian machines.

"The ITS command queue uses a little endian memory order model." (GIC Architecture Specification)

Make intr affinity work with MSIs again

Since all ITS instances share a common LPI configuration table, used a
shared vmem arena to allocate pic irqs. Idea from FreeBSD.

Use kmem_zalloc for its state, fixes possible crash if a driver tries to
set affinity before a CPU is online.

Set msi_id to its_id

branches: 1.23.2;
Traiing whitespace

Use PCI_MSIX_"TBL"BIR_MASK instead of PCI_MSIX_"PBA"BIR_MASK for MSI-X table.
This is not a real bug because both macros have the same value.

branches: 1.21.2;
Cleanup ThunderX quirk and print cache / shareability details for ITT tables

Enable caching of ITS tables when possible

Restore original size calculation for MAPD and don't disable MSI/MSIX on devices before making changes

Rename lpi pic to gicv3-its when an ITS is found

Pass correct EventID to MOVI and INV commands

Remove unused variable (build fix)

Ensure that the "size" parameter of MAPD matches the size of the ITT being
mapped and subtract the LPI base from EventID. Fixes multi-vector MSI/MSI-X
on RK3399.

Pass correct event ID with MOVI commands

- Disable MSI/MSI-X when making changes
- MSI: Write the vector count to the Multi Message Enable (MME) field
- MSI: Set DATA to the first LPI number, not the last

Allow set_affinity calls before PEs are brought online. We store the
desired target PE if set_affinity is called early and restore the routes
when the PE comes alive.

Fail gracefully if gicv3_its_set_affinity is called before a cpu is
brought online.

branches: 1.10.4;
Add support for decoding PCI ID mappings using IO remapping tables (IORT).

Allow non-power of 2 counts, and support alloc/release/alloc patterns for a device as long as the ITT size is sufficient

branches: 1.8.2;
Handle ThunderX errata

Only allocate what we need for ITT tables

Fix ITT size, only need to MAPD once per device.

use correct interrupt index

fixes intrctl list

from jmcneill@

kcpuset_ffs returns the cpu number plus one, so make sure to subtract it

Add intr_establish_xname support to arm and expose it to intrctl

Implement pic_get_affinity/pic_set_affinity for LPIs via ITS

Add support for GICv3 Locality-specific Periphal Interrupts (LPIs) and the
Interrupt Translation Service (ITS).

gicv3_its: rename pt_dev_id to pt_index

gicv3: ITS: Add support for indirect device tables.

When devbits is too large to fit into the first level of device table,
some hardware supports the concept of indirect tables. Add support for
this and use it when supported.

This is required for MSIs to work properly on Snapdragon X Elite platforms
(otherwise, MAPD command fails and stalls the ITS command queue).

branches: 1.8.10;
Rework Arm (32bit and 64bit) AP startup so that cpu_hatch doesn't sleep.

The AP initialisation code in cpu_init_secondary_processor will read and
initialise the required system registers and state for the BP to attach
and report.

Rework the interrupt handler code for this new sequence. Thankfully,
this removes a bunch of code for bcm2836mp.

The VFP detection handler on <= armv7 relies on the global undefined
handler being in place until the BP attaches vfp. That is, after the
APs have been spun up.

gicv3_its.c has a serialisation issue which is protected against in
the gicv3_its_cpu_init, which is called from cpu_hatch, with a spin
lock. The serialisation issue needs addressing more completely.

Tested on RPI3, Apple M1, QEMU, and lx2k

Fixes PR port-arm/56264:
diagnostic assertion "l->l_stat == LSONPROC" failed on RPI3

Remove MAXCPUS dependency.

branches: 1.6.10;
Allow set_affinity calls before PEs are brought online. We store the
desired target PE if set_affinity is called early and restore the routes
when the PE comes alive.

Fail gracefully if gicv3_its_set_affinity is called before a cpu is
brought online.

branches: 1.4.4;
Allow non-power of 2 counts, and support alloc/release/alloc patterns for a device as long as the ITT size is sufficient

branches: 1.3.2;
include dev/pci/pcivar.h, we use pci_attach_args directly

Implement pic_get_affinity/pic_set_affinity for LPIs via ITS

Add support for GICv3 Locality-specific Periphal Interrupts (LPIs) and the
Interrupt Translation Service (ITS).

gtmr_delay: Put SPINLOCK_BACKOFF_HOOK in busy loop

It is expanded as a `yield` instruction for aarch64.

This slightly improves peak performance for some drivers on
MP environments, as tested by msaitoh@ (thanks!).

XXX
This is what OpenBSD does, while FreeBSD just busy-spins.

However, unfortunately, benefits from `yield` should be limited,
as we still read count register quite frequently.

For long-duration delays, Linux uses `wfit` and `wfet` insns if
provided, and falls back to `wfe` otherwise:

https://github.com/torvalds/linux/blob/master/arch/arm64/lib/delay.c#L26-L51

Event streams should be configured for this purpose.

branches: 1.49.10;
arm: Use device_set_private for various drivers.

fix aarch64 build

gtmr: Add support for arm,cpu-registers-not-fw-configured property.

On armv7, arm,cpu-registers-not-fw-configured means that firmware hasn't
bothered to configure any generic timer registers and we need to
initialize cntfrq ourselves.

Rework Arm (32bit and 64bit) AP startup so that cpu_hatch doesn't sleep.

The AP initialisation code in cpu_init_secondary_processor will read and
initialise the required system registers and state for the BP to attach
and report.

Rework the interrupt handler code for this new sequence. Thankfully,
this removes a bunch of code for bcm2836mp.

The VFP detection handler on <= armv7 relies on the global undefined
handler being in place until the BP attaches vfp. That is, after the
APs have been spun up.

gicv3_its.c has a serialisation issue which is protected against in
the gicv3_its_cpu_init, which is called from cpu_hatch, with a spin
lock. The serialisation issue needs addressing more completely.

Tested on RPI3, Apple M1, QEMU, and lx2k

Fixes PR port-arm/56264:
diagnostic assertion "l->l_stat == LSONPROC" failed on RPI3

If we get a spurious interrupt, log a debug message and ignore it.

Otherwise we risk tripping an assertion later on due to an interrupt
firing before it is scheduled.

- Add an isb before cnt[pv]_ctl read
- cntk_ctl is 64-bits
- Do not toggle CNTCTL_IMASK in intr handler, as this may be needed
elsewhere.

Fix a comment

branches: 1.42.2;
Retire arm_[di]sb in favour of the isb() and dsb(sy) macro invocations.

Add support for physical timers and sprinkle isb where needed.

branches: 1.40.2;
Disable counter before updating cval/tval

Fix bitmask in A64 errata workaround and write cval instead of tval on A64

A64 timer errata can cause the timer to fire too soon, so skip KASSERT here too.

Disable diagnostic assertions around timer jitter on Allwinner A64. It
seems the instability of CNTVCT can cause issues with the compare value,
and rather than applying a heavy workaround just skip the panics.

If we're going to allow pl0 to access virtual and physical counters then
we should allow it from all CPUs.

kern/53630 (openssl fallout on arm)

port-evbarm/53308: evbarm-earmv7hf performance regression under qemu

Revert gtmr.c r1.27 and apply a workaround for Allwinner A64 SoCs based
on analysis of the issue from LKML: https://lkml.org/lkml/2018/5/10/774

Since this bug is specific to the Allwinner A64 SoC, only apply the
workaround when the root ("/") node of the device tree is compatible
with "allwinner,sun50i-a64".

Typo in comment from Joerg's gtmr diff

fix non-DIAGNOSTIC build

ARM ARM refers to this as "ARM Generic Timer", so adjust printf at attach
to match.

branches: 1.31.2;
fixed a case of gtmr_cntvct_stable_read() is returning the same value as before.

Simplify gtmr_delay and don't mix and match usage of the physical and
virtual timers (always use the virtual timer).

Avoid unnecessarily touching CNTP_CTL.

We may not have the privilege of accessing CNTP_CTL if running as a
virtualized guest, and we're not using the Physical Timer for interupt
generation anyway.

Replace stable_read/write debug printfs with event counters

Workaround A-008585 errata in GTMR.

Register reads and writes may provide unstable results if the counter
hardware is active at the same time. This results in non-monotonic
counters seen by both the gtmr interrupt and time counter.

The loops are currently applied unconditionally, restricting them to
appropiate FDT markers can be applied later.

Remove a number of debug #if 0s.

Remove unused gtmr_bootdelay.

Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)

branches: 1.23.2;
When attaching gtmr at fdt use fdtbus_intr_establish to establish the
gtmr interrupt.

Improve delay to be better than 1.3 innaccurate by default and handle the
unlikely wrap

Trailing whitespace

gtmr_intr: If the ISTATUS bit is 0, timer condition is not met. In this
case, just return 0.

Add __HAVE_GENERIC_CPU_INITCLOCKS option. If set, don't export
cpu_initclocks from device drivers as common code (in this case FDT) will
provide its own copy.

Interrupts are enabled before the timer is configured. Ensure that the
timer is disabled when attaching so it doesn't go crazy between the time
interrupts are enabled and clocks are initialized. My RPI3 makes it
multi-user now.

branches: 1.17.2;
If the "frequency" property is not specified, try to use the value in
CNTFRQ. Firmware should have already set this for us.

Use physical timer for gtmr_delay();

Don't adjust interval if we are too far into the next tick.

Whitespace

Trailing whitespace.

Move isb's to before reading count. Change interrupt to level.

Sprinkle some arm_isb() to force execution of the mcrr instructions to get
more accurate values. (Given that the generic timer is usually running at
a 64th or less of the cpu clock, this only settle to a tick or so.)

port-arm/49737: armgtmr0 timer broken

Use physical timer instead of virtual timer for timecounter. For platforms
that implement virtualization extensions, the CNTVOFF register defines a
virtual offset between the physical count and virtual count. Unfortunately,
the CNTVOFF register is only accessible in secure mode and the value is
per-CPU, so we may end up in a scenario where virtual count reads from
CPU A -> B -> A are not monotonic. No offset applied to physical timer,
so physical count reads are guaranteed to be monotonic.

Initial RPI2 support - it doesn't work yet. The generic timer gets messed
up somehow.

This commit changes the KVA layout of the RPI.

branches: 1.8.2; 1.8.4;
Add a KASSERT to make sure the counter is running.

branches: 1.7.2;
Make sure freq is not 0.

branches: 1.6.2;
Add gtmr_bootdelay. rename clockhandler to gtmr_intr

branches: 1.5.2;
delta can't be negative. Wrapped value is seen as large positive, so
covered already.

Cleanup some diagnostic code.

Fix some bugs.

branches: 1.2.2; 1.2.4;
Pass the offset from CBAR/PERIPHBASE in mpcore_attach_args.
Modify the list of devices to include the offset(s) from PERIPHBASE.

Add generic timer support (untested)

branches: 1.1.2; 1.1.8; 1.1.12;
Add generic timer support (untested)

G/C gtmr_init function declaration

gtmr: Add support for arm,cpu-registers-not-fw-configured property.

On armv7, arm,cpu-registers-not-fw-configured means that firmware hasn't
bothered to configure any generic timer registers and we need to
initialize cntfrq ourselves.

Revert "Include opt_diagnostic.h for DIAGNOSTIC."

This did not do what I thought it did. opt_diagnostic.h is only for
the unused _DIAGNOSTIC, which seems like an abortive attempt to
incrementally convert DIAGNOSTIC to an opt_*.h option rather than a
command-line option.

Include opt_diagnostic.h for DIAGNOSTIC.

...at least, in header files, which may not have already included
libkern.h.

Add support for physical timers and sprinkle isb where needed.

branches: 1.11.4;
port-evbarm/53308: evbarm-earmv7hf performance regression under qemu

Revert gtmr.c r1.27 and apply a workaround for Allwinner A64 SoCs based
on analysis of the issue from LKML: https://lkml.org/lkml/2018/5/10/774

Since this bug is specific to the Allwinner A64 SoC, only apply the
workaround when the root ("/") node of the device tree is compatible
with "allwinner,sun50i-a64".

branches: 1.10.2;
Remove unused gtmr_bootdelay.

branches: 1.9.2;
When attaching gtmr at fdt use fdtbus_intr_establish to establish the
gtmr interrupt.

Trailing whitespace

Add __HAVE_GENERIC_CPU_INITCLOCKS option. If set, don't export
cpu_initclocks from device drivers as common code (in this case FDT) will
provide its own copy.

#include <sys/percpu.h>

Initial RPI2 support - it doesn't work yet. The generic timer gets messed
up somehow.

This commit changes the KVA layout of the RPI.

branches: 1.4.4; 1.4.6; 1.4.8;
Add gtmr_bootdelay. rename clockhandler to gtmr_intr

branches: 1.3.2;
Cleanup some diagnostic code.

branches: 1.2.2; 1.2.4;
Pass the offset from CBAR/PERIPHBASE in mpcore_attach_args.
Modify the list of devices to include the offset(s) from PERIPHBASE.

Add generic timer support (untested)

Initial RPI2 support - it doesn't work yet. The generic timer gets messed
up somehow.

This commit changes the KVA layout of the RPI.

branches: 1.2.6; 1.2.10; 1.2.12;
Pass the offset from CBAR/PERIPHBASE in mpcore_attach_args.
Modify the list of devices to include the offset(s) from PERIPHBASE.

branches: 1.1.2; 1.1.4; 1.1.6;
Add Cortex-A9 support including the ARM Generic Interrupt Controller
and the A9 Global Timer / Watchdog.

Pandaboard work from Rui-Xiang Guo via port-arm with updates from me.

https://github.com/picohive/netbsd-pandaboard

Rename arml2cc_init to arml2cc_get_cacheinfo.

NFCI.

spaces -> TAB

Trailing whitespace

Remove hard requirement for "offset" property on Cortex-A5. This is not required w/ FDT.

branches: 1.18.2;
Add l2cc support.

branches: 1.17.16;
allow arml2cc to be used on Cortex-A5 if the "offset" property is specified

clean the a9 l2 cache before turning it on.

branches: 1.15.2; 1.15.4;
Allow l2cc base to gotten from device properties.

branches: 1.14.2;
pl310 cache is PIPT

#include <arm/locore.h>

branches: 1.12.2; 1.12.6;
KASSERT -> KASSERTMSG

simplify cache range op

Don't "sync" atomic ops.
Do sync after each range op.

Make these compile with gcc4.1 and binutils 2.16

branches: 1.8.2;
Invalidate the L2 cache before enabling it.

branches: 1.7.2;
Add a missing mutex exit.

Don't use an asm in pmap_activate to update the TTBR, use cpu_setttb instead
but add a second argument to it to indicate whether the TLB/caches need to be
flushed. Default cortex to pmap_needs_fixup = 1. But check the MMFR3 field
to see if the fixed can be skipped.
Use a cf_flag bit 0 to indicate whether the A9 L2 cache should disable (bit 0 = 1)
or enabeld (bit = 0).

With these changes, the A9 MMU can use traverse caches to do MMU tablewalks
Also, make sure all memory has the shareable bit for the A9.

Add L2 cache flush routines. (not yet enabled).

branches: 1.4.2;
Don't disable the L2C is it isn't enabled.

Switch cortex_a9 back to need_ptesync = 1
Add code to disable the L2 cache on cortex-a9 (for now).
Add evcnt for all the fault types.
Move cache info in a structure and have one for the pcache and one for scache.
Probe L1/L2 caches properly for ARMv7

Add "write-back" before Unified

Add driver to attach ARM PL210 L2 Cache Controller
arml2cc0 at armperiph0: ARM PL310 L2 r3p2 Cache Controller
arml2cc0: 256KB/32B 16-way L2 Unified cache

Add constant for a reserved bit 31 for the L2C_AUXCTL register for use in
masks

Add defines for the PL310's L2C_PREFETCH_CTL and L2C_POWER_CTL registers as
per r3p2 spec.

branches: 1.2.2; 1.2.4; 1.2.6; 1.2.8; 1.2.16;
Switch cortex_a9 back to need_ptesync = 1
Add code to disable the L2 cache on cortex-a9 (for now).
Add evcnt for all the fault types.
Move cache info in a structure and have one for the pcache and one for scache.
Probe L1/L2 caches properly for ARMv7

Add driver to attach ARM PL210 L2 Cache Controller
arml2cc0 at armperiph0: ARM PL310 L2 r3p2 Cache Controller
arml2cc0: 256KB/32B 16-way L2 Unified cache

Pandaboard work from Rui-Xiang Guo via port-arm with updates from me.

https://github.com/picohive/netbsd-pandaboard

Rename arml2cc_init to arml2cc_get_cacheinfo.

NFCI.

branches: 1.1.4; 1.1.6; 1.1.16;
Switch cortex_a9 back to need_ptesync = 1
Add code to disable the L2 cache on cortex-a9 (for now).
Add evcnt for all the fault types.
Move cache info in a structure and have one for the pcache and one for scache.
Probe L1/L2 caches properly for ARMv7

Move register definiton to header

branches: 1.2.14; 1.2.16;
Add SCU_SIZE.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.16; 1.1.22; 1.1.26;
Add Cortex-A9 support including the ARM Generic Interrupt Controller
and the A9 Global Timer / Watchdog.