Home | History | Annotate | Download | only in include
History log of /src/sys/arch/aarch64/include/cpufunc.h
RevisionDateAuthorComments
 1.28  30-Dec-2024  jmcneill aarch64: Allow for alternate cpu_idle() implementations
 1.27  07-Feb-2024  msaitoh Remove ryo@'s mail addresses.
 1.26  20-Apr-2023  skrll Provide a shared pmap_devmap implementation and convert all pmap_devmap
arrays to use DEVMAP_ENTRY{,_END}
 1.25  10-Sep-2022  rillig fix misspellings of 'available' and nearby typos
 1.24  20-Jul-2022  riastradh aarch64: Make cpufunc.h includable without sys/cpu.h first.
 1.23  31-Jan-2022  ryo add support Hardware updates to Access flag and Dirty state (FEAT_HAFDBS)

- The DBM bit of the PTE is now used to determine if it is writable, and
the AF bit is treated entirely as a reference bit. A valid PTE is always
treated as readable. There can be no valid PTE that is not readable.
- LX_BLKPAG_OS_{READ,WRITE} are used only for debugging purposes,
and has been superseded by LX_BLKPAG_AF and LX_BLKPAG_DBM.
- Improve comment

The need for reference/modify emulation has been eliminated,
and access/permission faults have been reduced, however,
there has been little change in overall performance.
 1.22  31-Oct-2021  skrll Rework Arm (32bit and 64bit) AP startup so that cpu_hatch doesn't sleep.

The AP initialisation code in cpu_init_secondary_processor will read and
initialise the required system registers and state for the BP to attach
and report.

Rework the interrupt handler code for this new sequence. Thankfully,
this removes a bunch of code for bcm2836mp.

The VFP detection handler on <= armv7 relies on the global undefined
handler being in place until the BP attaches vfp. That is, after the
APs have been spun up.

gicv3_its.c has a serialisation issue which is protected against in
the gicv3_its_cpu_init, which is called from cpu_hatch, with a spin
lock. The serialisation issue needs addressing more completely.

Tested on RPI3, Apple M1, QEMU, and lx2k

Fixes PR port-arm/56264:
diagnostic assertion "l->l_stat == LSONPROC" failed on RPI3
 1.21  23-Oct-2021  skrll Whitespace
 1.20  27-May-2021  ryo fix build error with options ARMV85_BTI
 1.19  04-Dec-2020  skrll branches: 1.19.4; 1.19.6;
Remove unnecessary casts
 1.18  03-Aug-2020  ryo branches: 1.18.2;
Implement MD ucas(9) (__HAVE_UCAS_FULL)
 1.17  02-Aug-2020  maxv Add support for Privileged Access Never (ARMv8.1-PAN).

PAN provides the same functionality as SMAP on x86: it forbids kernel
access to userland pages when PSTATE.PAN=1, and allows such accesses when
PSTATE.PAN=0.

We clear SCTLR_SPAN, to guarantee that PAN=1 each time the kernel is
entered. We catch PAN faults and panic right away without further
processing. In copyin, copyout, etc, we temporarily authorize access to
userland pages.

PAN is a very useful exploit mitigation. Reviewed by ryo@, thanks. Tested
on Qemu. Enabled by default.
 1.16  01-Jul-2020  ryo Switch the Icache sync operation to the necessary and sufficient one according to the CTR_EL0.DIC and CTR_EL0.IDC flags.

If CTR_EL0.DIC=1, Icache invalidation is not required.
If CTR_EL0.IDC=1, Dcache clean before Icache invalidation is not required.
CLIDR_EL1.LoC is 0, or CLIDR_EL1.LoUIS and CLIDR_EL1.LoUU are 0, Dcache clean is not required as well.

SEE ALSO ARMARM, "CTR_EL0 Cache Type Register", and "CLIDR_EL1 Cache Level ID Register"
 1.15  25-May-2020  ryo cache information can be detected correctly on newer CPUs

- add VPIPT cache type
- adapt to 64-bit CCSIDR (ARMv8.3-CCIDX)
- CCSIDR:[WT,WB,PA,WA] are deprecated
- show number of cache lines when attaching cpu
 1.14  15-May-2020  ryo SCTLR_EnIA should be enabled in the caller(locore).

For some reason, gcc make aarch64_pac_init() function non-leaf, and it uses paciasp/autiasp.
 1.13  13-May-2020  ryo - move aarch64 addressspace macros from pmap.h to cpufunc.h
- rename ptr_strip_pac() to aarch64_strip_pac()
 1.12  12-Apr-2020  maxv Add support for Pointer Authentication (PAC).

We use the "pac-ret" option, to sign the return instruction pointer on
function entry, and authenticate it on function exit. This acts as a
mitigation against ROP.

The authentication uses a per-lwp (secret) I-A key stored in the 128bit
APIAKey register and part of the lwp context. During lwp creation, the
kernel generates a random key, and during context switches, it installs
the key of the target lwp on the CPU.

Userland cannot read the APIAKey register directly. However, it can sign
its pointers with it, because the register is architecturally shared
between userland and the kernel. Although part of the CPU design, it is
a bit of an undesired behavior, because it allows to forge valid kernel
pointers from userland. To avoid that, we don't share the key with
userland, and rather switch it in EL0<->EL1 transitions. This means that
when userland executes, a different key is loaded in APIAKey than the one
the kernel uses. For now the userland key is a fixed 128bit zero value.

The DDB stack unwinder is changed to strip the authentication code from
the pointers in lr.

Two problems are known:

* Currently the idlelwps' keys are not really secret. This is because
the RNG is not yet available when we spawn these lwps. Not overly
important, but would be nice to fix with UEFI RNG.
* The key switching in EL0<->EL1 transitions is not the most optimized
code on the planet. Instead of checking aarch64_pac_enabled, it would
be better to hot-patch the code at boot time, but there currently is
no hot-patch support on aarch64.

Tested on Qemu.
 1.11  15-Jan-2020  mrg branches: 1.11.4;
port the arm64 cpu topology setup for big.little to arm.

rename arm64 cpu_do_topology() to arm_cpu_do_topology() and
call it from both arm cpu_attach().

replace both aarch64_set_topology() inline code in arm
cpu_attach() with new arm_cpu_do_topology(), which is called
by the arm64 locore as well (possibly not needed, which would
allow it to become static.)

not yet tested on a real big.little armv7 system. tested
on rockpro64 and pinebook pro.
 1.10  12-Jan-2020  mrg provide some semblance of valid cpu topology for big.little systems.

while attaching cpus, if the FDT provides "capacity-dmips-mhz" track
the fastest set, and call cpu_topology_set() with slow=true for any
cpus that are not the fastest.

bug fix for cpu_topology_set(): actually set ci_is_slow for slow cpus.

with this change, and -current's recent scheduler changes, this means
that long running processes run on the faster cores. on RK3399 based
systems, i am seeing 20-50% speed ups for many tasks.


XXX: all this can be made common with armv7 big.little.
 1.9  19-Dec-2019  ryo branches: 1.9.2;
aarch64_cache_info[] is not global
 1.8  22-Nov-2019  mlelstv Make cache operations available early.
 1.7  13-Sep-2019  ryo In pmap_devmap_bootstrap(), cpu_earlydevice_va_p() must not return true until *all* devmap tables have been enabled.
console mapping may be present in the last table.
 1.6  07-Sep-2019  ryo add checking status of MMU and devmap to make _platform_early_putchar() available at all times.
 1.5  21-Dec-2018  ryo - add workaround for Cavium ThunderX errata 27456.
- add cpufuncs table in cpu_info. each cpu clusters may have different erratum. (e.g. big.LITTLE)
 1.4  15-Dec-2018  alnsn Add missing include for device_t declaration.
 1.3  26-Aug-2018  ryo add support multiple cpu clusters.
* pass cpu index as an argument to secondary processors when hatching.
* keep cpu cache confituration per cpu clusters.

Hello big.LITTLE!
 1.2  23-Jul-2018  ryo * fix icache invalidations.
* "ic ivau" (aarch64_icache_sync_range) with VA generates permission fault in some situations, therefore use KSEG address for now.
 1.1  01-Apr-2018  ryo branches: 1.1.2; 1.1.4;
Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@)

- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
 1.1.4.4  21-Apr-2020  martin Sync with HEAD
 1.1.4.3  13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.1.4.2  08-Apr-2020  martin Merge changes from current as of 20200406
 1.1.4.1  10-Jun-2019  christos Sync with HEAD
 1.1.2.5  26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.1.2.4  06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.1.2.3  28-Jul-2018  pgoyette Sync with HEAD
 1.1.2.2  07-Apr-2018  pgoyette Sync with HEAD. 77 conflicts resolved - all of them $NetBSD$
 1.1.2.1  01-Apr-2018  pgoyette file cpufunc.h was added on branch pgoyette-compat on 2018-04-07 04:12:11 +0000
 1.9.2.1  17-Jan-2020  ad Sync with head.
 1.11.4.1  20-Apr-2020  bouyer Sync with HEAD
 1.18.2.1  14-Dec-2020  thorpej Sync w/ HEAD.
 1.19.6.1  31-May-2021  cjep sync with head
 1.19.4.1  17-Jun-2021  thorpej Sync w/ HEAD.

RSS XML Feed