| History log of /src/sys/arch/arm/cortex/gicv3.c |
| Revision | | Date | Author | Comments |
| 1.56 |
| 04-Sep-2025 |
rin | gicv3: Some more left-shift v.s. int fixes
Suggested by msaitoh@, thanks!
|
| 1.55 |
| 04-Sep-2025 |
rin | gicv3: Appease KUBSAN, otherwise NFC
|
| 1.54 |
| 26-Jun-2022 |
jmcneill | branches: 1.54.4; 1.54.12;
build fix: remove includes of opt_gic.h
|
| 1.53 |
| 25-Jun-2022 |
jmcneill | gicv3: Treat all IPLs below IPL_VM as IPL_NONE for hwpl mappings
|
| 1.52 |
| 25-Jun-2022 |
jmcneill | Remove GIC_SPLFUNCS.
|
| 1.51 |
| 25-Jun-2022 |
jmcneill | pic: Update ci_cpl in pic_set_priority callback.
Not all ICs need interrupts disabled to update the priority. DAIF accesses
are not cheap, so push the update of ci_cpl from pic_set_priority to the
IC's pic_set_priority callback, and let the IC driver determine whether
or not it needs interrupts disabled.
|
| 1.50 |
| 28-Mar-2022 |
riastradh | arm/cortex: Use container_of, not bespoke offsetof arithmetic.
|
| 1.49 |
| 02-Oct-2021 |
skrll | Trailing whitespace
|
| 1.48 |
| 26-Sep-2021 |
jmcneill | If an SGI or PPI is established after interrupts are enabled, make sure
we unblock the source on _all_ CPUs and not just the CPU that is
establishing the interrupt.
|
| 1.47 |
| 11-Sep-2021 |
jmcneill | GIC: Support SPI affinity routing before secondary CPUs have hatched.
Instead of reading MPIDR from CPU init to determine a routing target, trust
system configuration data (Devicetree or ACPI) to properly fill in ci_cpuid.
This allows for SPI affinity routing to work before secondary CPUs are
hatched; previously, one of two things would happen:
a) If 1ofN distribution is supported, the SPI could be distributed to any
participating PE.
b) If 1ofN distribution is not supported, the SPI would not be forwarded
and would stay pending.
|
| 1.46 |
| 10-Aug-2021 |
jmcneill | Make gic_splfuncs optional and disable it by default until it has had
more testing.
|
| 1.45 |
| 10-Aug-2021 |
jmcneill | Use custom spl funcs for GIC and avoid unnecessary pmr register accesses
in splx.
|
| 1.44 |
| 28-Mar-2021 |
jmcneill | Disable 1ofN distribution of SPIs by default. This is a workaround for an
issue in the USB stack -- signaling transfer complete on multiple PEs can
cause transfer completions to be processed out of order.
|
| 1.43 |
| 23-Feb-2021 |
jmcneill | branches: 1.43.2;
If we are committing a deferred splhigh() to hardware, no need to continue.
|
| 1.42 |
| 21-Feb-2021 |
jmcneill | Keep current hardware priority value in struct cpu_info and use it instead
of reading icc_pmr_el1 in gicv3_set_priority.
|
| 1.41 |
| 09-Feb-2021 |
ryo | fix build without MULTIPROCESSOR
|
| 1.40 |
| 07-Feb-2021 |
jmcneill | Optimize hardware priority updates.
In gicv3_set_priority, read the current priority mask and only update it
if we are lowering the priority. The hardware priority filter is raised
only after taking an interrupt. This makes splfoo/splx without an interrupt
in between considerably cheaper as PMR writes are relatively expensive
compared to reads.
While here, avoid an extra daif read when dispatching interrupts by using
ENABLE_INTERRUPT() / DISABLE_INTERRUPT() instead of cpsie() / cpsid() macros.
|
| 1.39 |
| 16-Jan-2021 |
jmcneill | Remove MAXCPUS dependency.
|
| 1.38 |
| 22-Dec-2020 |
jmcneill | When lpiconf flushing is required, make sure to flush to PoC and not PoU.
Spotted by nick.
|
| 1.37 |
| 11-Dec-2020 |
jmcneill | Fix spelling in comment.
|
| 1.36 |
| 04-Dec-2020 |
jmcneill | gicv3: Only use 1 of N SPI distribution when the feature is available.
A GICv3+ implementation is not guaranteed to support 1 of N SPI
distribution. Support for this feature is indicated in GICD_TYPER.No1N.
When No1N=1, route all interrupts to the primary PE by default and only
allow a single CPU target when updating affinity.
|
| 1.35 |
| 24-Nov-2020 |
jmcneill | Improve detection of NS vs S views of priorities.
For PMR, write a 0 to bit7 and see if it sticks. This is only possible from
NS EL1 if we have a non-secure view of ICC_PMR_EL1.
For int priorities (GICD/GICR interfaces and LPIs), assume that the
GICD_CTLR.DS bit is telling us the truth.
RK3399 is special here when using the vendor bootloader, so keep the
auto-detection from the previous commit but limit the scope to only run
on RK3399 SOCs.
|
| 1.34 |
| 22-Nov-2020 |
jmcneill | Fix interrupt priorities on N1 SDP.
The GICv3 architecture specification is not clear on the NS view of
priority registers, and there doesn't seem to be any consistency in how
these are implemented in both real and emulated environments.
The previous fix for this issue was meant to detect what we thought at the
time was a bug on the Rockchip RK3399. At that time the theory was somehow
EL1 has a secure view of the hardware, and this is causing us to have the
wrong view of IPRIORITYRn based on IHI0069F section 4.8.6 "Software
accesses of interrupt priority". But it turns out that this is not the
full picture. While I was able to confirm that yes, we do have secure
access to the GIC on RK3399 from EL1, the view of IPRIORITYRn differs
depending on whether you are using the Rockchip TF-A as included with
https://github.com/ayufan-rock64/linux-u-boot (shifted view), or mainline
TF-A from pkgsrc (unshifted view).
So to detect this quirk, we need three things: A method to detect if we
have S access to GIC registers, a method to see how many PMR bits are
implemented, and a method to see how many IPRIORITYRn bits are implemented.
To detect S access, we can try to toggle GICD_CTRL.EnableGrp1S. This bit
is either RES0 (security extensions not implemented), RAZ/WI (non-secure
access in two security state systems) or RW (secure access in two security
state systems).
To read the number of PMR and IPRIORITYRn bits supported, we can write all
1s to the register fields and read them back.
For the RK3399 (Rockchip TF-A) quirk, we assume a shifted view of
IPRIORITYRn if we have detected S accesses, and the PMR and IPRIORITYRn
values differ. The S access test is required because some real hardware
implementations (Ampere eMAG) were observed to report different PMR and
IPRIORITYRn masks, but present an unshifted view of IPRIORITYRn.
During testing, I also discovered that QEMU 5.1 requires this shifted view
workaround as well -- as far as I can tell, this is a QEMU bug. We can't
detect it the same way as RK3399 because security is disabled in the
emulated GIC, and the PMR and IPRIORITYRn tests both return 0xff! So now
if the GICv3 driver sees this configuration, it assumes that the shifted
view is required.
Honestly, this feature is so poorly documented that maybe it is better to
give up on HW priorities and preemption and use a single flat model like
Linux and FreeBSD does.
Tested on Arm N1 SDP, ROCKpro64 (RK3399) with Rockchip and pkgsrc TF-A,
Pinebook Pro (RK3399), Lenovo HR330A (Ampere eMAG), QEMU 5.1 (gic-version=3),
AWS EC2 a1.medium (Graviton).
|
| 1.33 |
| 21-Nov-2020 |
jmcneill | Add a per-CPU event counter that counts every time an interrupt handler is
preempted by a higher priority interrupt.
|
| 1.32 |
| 01-Nov-2020 |
jmcneill | branches: 1.32.2;
sc_enabled_sgippi can be updated from any CPU, so use atomic_or/atomic_and
|
| 1.31 |
| 01-Nov-2020 |
jmcneill | Remove unused __HAVE_PIC_FAST_SOFTINTS block. It never would have worked
if enabled..
|
| 1.30 |
| 01-Nov-2020 |
jmcneill | Add an isb() barrier after ICC_SGI1R_EL1 write to prevent reordering with
subsequent wfi/wfe instructions. Haven't seen this in practice but I would
rather be safe here.
|
| 1.29 |
| 01-Nov-2020 |
jmcneill | gicv3_set_priority: ICC_PMR_EL1 is self-synchronizing so no need for isb()
here.
|
| 1.28 |
| 01-Nov-2020 |
jmcneill | gicv3_irq_handler: No need to call gicv3_set_priority if we are already at
the desired ipl.
|
| 1.27 |
| 01-Nov-2020 |
jmcneill | gicv3_ipi_send: simplify logic in kcp != NULL case given that we know that
the kcpuset will only ever contain one cpu.
|
| 1.26 |
| 30-Oct-2020 |
skrll | Retire arm_[di]sb in favour of the isb() and dsb(sy) macro invocations.
|
| 1.25 |
| 13-Apr-2020 |
jmcneill | Apply similar fix from gic.c that fixed "left shift of 255 by 24 places
cannot be represented in type 'int'" warnings from UBSan.
|
| 1.24 |
| 13-Feb-2020 |
jmcneill | branches: 1.24.4;
Make intr affinity work with MSIs again
|
| 1.23 |
| 13-Feb-2020 |
jmcneill | Since all ITS instances share a common LPI configuration table, used a
shared vmem arena to allocate pic irqs. Idea from FreeBSD.
|
| 1.22 |
| 24-Dec-2019 |
skrll | branches: 1.22.2;
Traiing whitespace
|
| 1.21 |
| 05-Sep-2019 |
jmcneill | - Use pic_do_pending_ints in intr handler
- Sprinkle isb
- Fix PMR bits detection on eMAG, from OpenBSD
|
| 1.20 |
| 30-Jun-2019 |
jmcneill | branches: 1.20.2;
Fix size of LPI pending table allocation and enable caching of LPI conf
and pending tables where possible.
|
| 1.19 |
| 26-Jun-2019 |
jmcneill | Change how we detect secure vs non-secure access.
Write 0xff to ICC_PMR_EL1 and read back how many bits are implemented,
then do the same with a GICD_IPRIORITYR<n> priority value field.
If the values differ, assume we have a shifted view of IPRIORITYR.
|
| 1.18 |
| 17-Jun-2019 |
jmcneill | Improve priority handling for cases where access is secure, from OpenBSD.
|
| 1.17 |
| 12-Jun-2019 |
mrg | revert rev 1.4:
>Adjust priority mappings, NFCI
it has some unintended change that makes nvme hangy. ok @jmcneill.
|
| 1.16 |
| 12-Jun-2019 |
jmcneill | Revert "Route all interrupts to the primary PE by default"
|
| 1.15 |
| 12-Jun-2019 |
jmcneill | Route all interrupts to the primary PE by default
|
| 1.14 |
| 12-Jun-2019 |
jmcneill | Adjust priority mappings, NFCI
|
| 1.13 |
| 23-Nov-2018 |
jmcneill | branches: 1.13.4;
Fix LPI pending table size, use correct LPI conf offset in gicv3_lpi_block_irqs, and set bit[7]=1 for G1NS interrupts when writing to the LPI configuration table.
|
| 1.12 |
| 21-Nov-2018 |
jmcneill | kcpuset_ffs returns the cpu number plus one, so make sure to subtract it
|
| 1.11 |
| 17-Nov-2018 |
jmcneill | Use intr_establish_xname
|
| 1.10 |
| 15-Nov-2018 |
jmcneill | Instead of disabling preemption, set the binary point field to the minimum supported value
|
| 1.9 |
| 13-Nov-2018 |
jmcneill | Update GICD_CTLR reg bit definitions to reflect the layout of the register
when either in non-secure state or for a system that only supports a single
state.
|
| 1.8 |
| 13-Nov-2018 |
jmcneill | Save a few pages by only allocating LPI pending tables for "ncpu" instead of "MAXCPU" CPUs.
|
| 1.7 |
| 10-Nov-2018 |
jmcneill | Implement pic_get_affinity/pic_set_affinity for LPIs via ITS
|
| 1.6 |
| 10-Nov-2018 |
jmcneill | Implement pic_get_affinity/pic_set_affinity for SPIs
|
| 1.5 |
| 09-Nov-2018 |
jmcneill | Add support for GICv3 Locality-specific Periphal Interrupts (LPIs) and the
Interrupt Translation Service (ITS).
|
| 1.4 |
| 05-Nov-2018 |
jmcneill | ICC_PMR_EL1 has different encoding than IPRIORITYR. Not 100% sure that this is correct yet, but it works with both RK3399 and QEMU.
|
| 1.3 |
| 30-Sep-2018 |
jmcneill | Set NS access bit when writing ICC_PMR_EL1
|
| 1.2 |
| 11-Aug-2018 |
jmcneill | branches: 1.2.2;
IPI and priority fixes. My RK3399 board boots multiuser now.
|
| 1.1 |
| 08-Aug-2018 |
jmcneill | Add GICv3 support.
|
| 1.2.2.4 |
| 26-Nov-2018 |
pgoyette | Sync with HEAD, resolve a couple of conflicts
|
| 1.2.2.3 |
| 20-Oct-2018 |
pgoyette | Sync with head
|
| 1.2.2.2 |
| 06-Sep-2018 |
pgoyette | Sync with HEAD
Resolve a couple of conflicts (result of the uimin/uimax changes)
|
| 1.2.2.1 |
| 11-Aug-2018 |
pgoyette | file gicv3.c was added on branch pgoyette-compat on 2018-09-06 06:55:26 +0000
|
| 1.13.4.5 |
| 21-Apr-2020 |
martin | Sync with HEAD
|
| 1.13.4.4 |
| 13-Apr-2020 |
martin | Mostly merge changes from HEAD upto 20200411
|
| 1.13.4.3 |
| 08-Apr-2020 |
martin | Merge changes from current as of 20200406
|
| 1.13.4.2 |
| 10-Jun-2019 |
christos | Sync with HEAD
|
| 1.13.4.1 |
| 23-Nov-2018 |
christos | file gicv3.c was added on branch phil-wifi on 2019-06-10 22:05:52 +0000
|
| 1.20.2.1 |
| 22-Sep-2019 |
martin | Pull up following revision(s) (requested by jmcneill in ticket #222):
sys/arch/arm/cortex/gicv3.c: revision 1.21
- Use pic_do_pending_ints in intr handler
- Sprinkle isb
- Fix PMR bits detection on eMAG, from OpenBSD
|
| 1.22.2.1 |
| 29-Feb-2020 |
ad | Sync with head.
|
| 1.24.4.1 |
| 20-Apr-2020 |
bouyer | Sync with HEAD
|
| 1.32.2.3 |
| 03-Apr-2021 |
thorpej | Sync with HEAD.
|
| 1.32.2.2 |
| 03-Jan-2021 |
thorpej | Sync w/ HEAD.
|
| 1.32.2.1 |
| 14-Dec-2020 |
thorpej | Sync w/ HEAD.
|
| 1.43.2.1 |
| 03-Apr-2021 |
thorpej | Sync with HEAD.
|
| 1.54.12.1 |
| 05-Sep-2025 |
martin | Pull up following revision(s) (requested by rin in ticket #25):
sys/arch/arm/cortex/gicv3.c: revision 1.55
sys/arch/arm/cortex/gicv3.c: revision 1.56
gicv3: Appease KUBSAN, otherwise NFC
gicv3: Some more left-shift v.s. int fixes
Suggested by msaitoh@, thanks!
|
| 1.54.4.1 |
| 05-Sep-2025 |
martin | Pull up following revision(s) (requested by rin in ticket #1156):
sys/arch/arm/cortex/gicv3.c: revision 1.55
sys/arch/arm/cortex/gicv3.c: revision 1.56
gicv3: Appease KUBSAN, otherwise NFC
gicv3: Some more left-shift v.s. int fixes
Suggested by msaitoh@, thanks!
|
