Home | History | Annotate | Download | only in x86
History log of /src/sys/arch/xen/x86/xenfunc.c
RevisionDateAuthorComments
 1.29  20-Aug-2022  riastradh x86: Split most of pmap.h into pmap_private.h or vmparam.h.

This way pmap.h only contains the MD definition of the MI pmap(9)
API, which loads of things in the kernel rely on, so changing x86
pmap internals no longer requires recompiling the entire kernel every
time.

Callers needing these internals must now use machine/pmap_private.h.
Note: This is not x86/pmap_private.h because it contains three parts:

1. CPU-specific (different for i386/amd64) definitions used by...

2. common definitions, including Xenisms like xpmap_ptetomach,
further used by...

3. more CPU-specific inlines for pmap_pte_* operations

So {amd64,i386}/pmap_private.h defines 1, includes x86/pmap_private.h
for 2, and then defines 3. Maybe we should split that out into a new
pmap_pte.h to reduce this trouble.

No functional change intended, other than that some .c files must
include machine/pmap_private.h when previously uvm/uvm_pmap.h
polluted the namespace with pmap internals.

Note: This migrates part of i386/pmap.h into i386/vmparam.h --
specifically the parts that are needed for several constants defined
in vmparam.h:

VM_MAXUSER_ADDRESS
VM_MAX_ADDRESS
VM_MAX_KERNEL_ADDRESS
VM_MIN_KERNEL_ADDRESS

Since i386 needs PDP_SIZE in vmparam.h, I added it there on amd64
too, just to keep things parallel.
 1.28  06-May-2020  bouyer xpq_queue_* use per-cpu queue; splvm() is enough to protect them.
remove the XXX SMP comments.
 1.27  25-Apr-2020  bouyer Merge the bouyer-xenpvh branch, bringing in Xen PV drivers support under HVM
guests in GENERIC.
Xen support can be disabled at runtime with
boot -c
disable hypervisor
 1.26  04-May-2019  kre branches: 1.26.8;

More of maxv's "switch to proper types" - hopefully unbreak i386 build.
 1.25  04-May-2019  maxv More inlined ASM. While here switch to proper types.
 1.24  06-Jan-2019  cherry Rollback http://mail-index.netbsd.org/source-changes/2018/12/22/msg101629.html

This change breaks module loading due to weak alias being unsupported
in the kernel module linker.

Requested by maxv@ and others as it affects their work.

No immediate decision on a replacement method is available, but other options
suggested include pre-processing, conditional compilation (#ifdef etc) and other
source level methods to avoid linktime decision making.
 1.23  22-Dec-2018  cherry Introduce a weak alias method of exporting different implementations
of the same API.

For eg: the amd64 native implementation of invlpg() now becomes
amd64_invlpg() with a weak symbol export of invlpg(), while the XEN
implementation becomes xen_invlpg(), also weakly exported as invlpg()

Note that linking in both together without having an override function
named invlpg() would be a mistake, as we have limited control over
which of the two options would emerge as the finally exported invlpg()
resulting in a potential situation where the wrong function is finally
exported. This change avoids this situation.

We should however include an override function invlpg() in that case,
such that it is able to then pass on the call to the appropriate
backing function (amd64_invlpg() in the case of native, and
xen_invlpg() in the case of under XEN virtualisation) at runtime.

This change does not introduce such a function and therefore does not
alter builds to include native as well as XEN implementations in the
same binary. This will be done later, with the introduction of XEN
PVHVM mode, where precisely such a runtime switch is required.

There are no operational changes introduced by this change.
 1.22  18-Oct-2018  cherry Zero out page table memory for IDT before use.
To copy the IDT entry before registration, de-reference the indexed
value, not the first entry.
Add a MAX_XEN_IDT value for max entries we expect and KASSERT() for
this as a sanity check.
 1.21  23-Sep-2018  cherry Fix for i386, functionality intended in:
http://mail-index.netbsd.org/source-changes/2018/09/23/msg099357.html

This should fix the build for both GENERIC and XEN3PAE_DOM0

This has not been boot tested on native or xen3pae

Notes: pmap_changeprot_local() seems to be x86_64 only.
I was a bit surprised by this initially, but I suspect that the table
protections are enforced via ring0/ring1 fencing rather than page protections

the gdt registration code in i386 is still messy. I will leave it as is
for now - to avoid a rabbit hole.
 1.20  23-Sep-2018  cherry Make XEN use the same api as native, for idt vector allocation
and registration.

lidt() placed in xenfunc() on maxv@ suggestion.

There should be no functional change due to this commit.

Tested on amd64 native and XEN.
 1.19  26-Jul-2018  maxv Remove dead code.
 1.18  24-Jun-2018  jdolecek branches: 1.18.2;
mark with XXXSMP all remaining spl*() and tsleep() calls
 1.17  15-Oct-2017  maxv branches: 1.17.2;
Add setusergs on Xen, and simplify.
 1.16  05-Feb-2017  maxv Rename ldt->ldtstore and gdt->gdtstore on i386. It reduces the diff with
amd64, and makes it easier to track down these variables on nxr - 'ldt'
and 'gdt' being common keywords.
 1.15  13-Dec-2016  kamil branches: 1.15.2;
Switch x86 CPU Debug Register types from vaddr_t to register_t

This is more opaque and appropriate type, as vaddr_t is meant to be used
for vitual address value. Not all DR on x86 are used to represent virtual
address (DR6 and DR7 are definitely not).

No functional change intended.

Change suggested by <christos>

Sponsored by <The NetBSD Foundation>
 1.14  27-Nov-2016  kamil Add accessors for available x86 Debug Registers

There are 8 Debug Registers on i386 (available at least since 80386) and 16
on AMD64. Currently DR4 and DR5 are reserved on both cpu-families and
DR9-DR15 are still reserved on AMD64. Therefore add accessors for DR0-DR3,
DR6-DR7 for all ports.

Debug Registers x86:
* DR0-DR3 Debug Address Registers
* DR4-DR5 Reserved
* DR6 Debug Status Register
* DR7 Debug Control Register
* DR8-DR15 Reserved

Access the registers is available only from a kernel (ring 0) as there is
needed top protected access. For this reason there is need to use special
XEN functions to get and set the registers in the XEN3 kernels.

XEN specific functions as defined in NetBSD:
- HYPERVISOR_get_debugreg()
- HYPERVISOR_set_debugreg()

This code extends the existing rdr6() and ldr6() accessor for additional:
- rdr0() & ldr0()
- rdr1() & ldr1()
- rdr2() & ldr2()
- rdr3() & ldr3()
- rdr7() & ldr7()

Traditionally accessors for DR6 were passing vaddr_t argument, while it's
appropriate type for DR0-DR3, DR6-DR7 should be using u_long, however it's
not a big deal. The resulting functionality should be equivalent so stick
to this convention and use the vaddr_t type for all DR accessors.

There was already a function defined for rdr6() in XEN, but it had a nit on
AMD64 as it was casting HYPERVISOR_get_debugreg() to u_int (32-bit on
AMD64), truncating result. It still works for DR6, but for the sake of
simplicity always return full 64-bit value.

New accessors duplicate functionality of the dr0() function available on
i386 within the KSTACK_CHECK_DR0 option. dr0() is a specialized layer with
logic to set appropriate types of interrupts, now accessors are designed to
pass verbatim values from user-land (with simple sanity checks in the
kernel). At the moment there are no plans to make possible to coexist
KSTACK_CHECK_DR0 with debug registers for user applications (debuggers).

options KSTACK_CHECK_DR0
Detect kernel stack overflow using DR0 register. This option uses DR0
register exclusively so you can't use DR0 register for other purpose
(e.g., hardware breakpoint) if you turn this on.

The KSTACK_CHECK_DR0 functionality was designed for i386 and never ported
to amd64.

Code tested on i386 and amd64 with kernels: GENERIC, XEN3_DOMU, XEN3_DOM0.

Sponsored by <The NetBSD Foundation>
 1.13  06-Nov-2011  cherry branches: 1.13.10; 1.13.28; 1.13.32;
[merging from cherry-xenmp] Make the xen MMU op queue locking api private. Implement per-cpu queues.
 1.12  13-Aug-2011  cherry branches: 1.12.2;
Add locking around ops to the hypervisor MMU "queue".
 1.11  24-Jul-2010  jym branches: 1.11.6;
Welcome PAE inside i386 current.

This patch is inspired by work previously done by Jeremy Morse, ported by me
to -current, merged with the work previously done for port-xen, together with
additionals fixes and improvements.

PAE option is disabled by default in GENERIC (but will be enabled in ALL in
the next few days).

In quick, PAE switches the CPU to a mode where physical addresses become
36 bits (64 GiB). Virtual address space remains at 32 bits (4 GiB). To cope
with the increased size of the physical address, they are manipulated as
64 bits variables by kernel and MMU.

When supported by the CPU, it also allows the use of the NX/XD bit that
provides no-execution right enforcement on a per physical page basis.

Notes:

- reworked locore.S

- introduce cpu_load_pmap(), used to switch pmap for the curcpu. Due to the
different handling of pmap mappings with PAE vs !PAE, Xen vs native, details
are hidden within this function. This helps calling it from assembly,
as some features, like BIOS calls, switch to pmap_kernel before mapping
trampoline code in low memory.

- some changes in bioscall and kvm86_call, to reflect the above.

- the L3 is "pinned" per-CPU, and is only manipulated by a
reduced set of functions within pmap. To track the L3, I added two
elements to struct cpu_info, namely ci_l3_pdirpa (PA of the L3), and
ci_l3_pdir (the L3 VA). Rest of the code considers that it runs "just
like" a normal i386, except that the L2 is 4 pages long (PTP_LEVELS is
still 2).

- similar to the ci_pae_l3_pdir{,pa} variables, amd64's xen_current_user_pgd
becomes an element of cpu_info (slowly paving the way for MP world).

- bootinfo_source struct declaration is modified, to cope with paddr_t size
change with PAE (it is not correct to assume that bs_addr is a paddr_t when
compiled with PAE - it should remain 32 bits). bs_addrs is now a
void * array (in bootloader's code under i386/stand/, the bs_addrs
is a physaddr_t, which is an unsigned long).

- fixes in multiboot code (same reason as bootinfo): paddr_t size
change. I used Elf32_* types, use RELOC() where necessary, and move the
memcpy() functions out of the if/else if (I do not expect sym and str tables
to overlap with ELF).

- 64 bits atomic functions for pmap

- all pmap_pdirpa access are now done through the pmap_pdirpa macro. It
hides the L3/L2 stuff from PAE, as well as the pm_pdirpa change in
struct pmap (it now becomes a PDP_SIZE array, with or without PAE).

- manipulation of recursive mappings ( PDIR_SLOT_{,A}PTEs ) is done via
loops on PDP_SIZE.

See also http://mail-index.netbsd.org/port-i386/2010/07/17/msg002062.html

No objection raised on port-i386@ and port-xen@R for about a week.

XXX kvm(3) will be fixed in another patch to properly handle both PAE and !PAE
kernel dumps (VA => PA macros are slightly different, and need proper 64 bits
PA support in kvm_i386).

XXX Mixing PAE and !PAE modules may lead to unwanted/unexpected results. This
cannot be solved easily, and needs lots of thinking before being declared
safe (paddr_t/bus_addr_t size handling, PD/PT macros abstractions).
 1.10  12-Feb-2010  jym branches: 1.10.2;
Starting with Xen 3 API, MMU_EXTENDED_COMMAND (tlb flush, cache flush, page
pinning/unpinning, set_ldt, invlpg) operations cannot be queued in the
xpq_queue[] any more, as they use their own specific hypercall, mmuext_op().

Their associated xpq_queue_*() functions already call xpq_flush_queue()
before issuing the mmuext_op() hypercall, which makes these xpq_flush_queue()
calls not necessary.

Rapidly discussed with bouyer@ in private mail. XEN3_DOM0/XEN3PAE_DOM0 tested
through a build.sh release, amd64 was only compile tested. No regression
expected.
 1.9  23-Oct-2009  snj branches: 1.9.2;
Remove 3rd and 4th clauses. OK cl@ (copyright holder).
 1.8  29-Jul-2009  cegger remove Xen2 support.
ok bouyer@
 1.7  11-May-2008  ad branches: 1.7.12;
Don't reload LDTR unless a new value, which only happens for USER_LDT.
 1.6  30-Apr-2008  cegger branches: 1.6.2;
AMD's APM Volume 2 says 'All control registers are 64bit in long mode'.
Fix the CR0 prototype to match this (the asm implementation is correct though).
OK ad
 1.5  21-Apr-2008  cegger branches: 1.5.2;
Access Xen's vcpu info structure per-CPU.
Tested on i386 and amd64 (both dom0 and domU) by me.
Xen2 tested (both dom0 and domU) by bouyer.
OK bouyer
 1.4  17-Feb-2008  bouyer branches: 1.4.6; 1.4.8;
Add missing __KERNEL_RCSID()
 1.3  11-Jan-2008  bouyer Merge the bouyer-xeni386 branch to head, at tag bouyer-xeni386-merge1 (the
branch is still active and will see i386PAE support developement).
Sumary of changes:
- switch xeni386 to the x86/x86/pmap.c, and the xen/x86/x86_xpmap.c
pmap bootstrap.
- merge back most of xen/i386/ to i386/i386
- change the build to reduce diffs between i386 and amd64 in file locations
- remove include files that were identical to the i386/amd64 counterparts,
the build will find them via the xen-ma/machine link.
 1.2  22-Nov-2007  bouyer branches: 1.2.2; 1.2.4; 1.2.8; 1.2.12; 1.2.16;
Pull up the bouyer-xenamd64 branch to HEAD. This brings in amd64 support
to NetBSD/Xen, both Dom0 and DomU.
 1.1  17-Oct-2007  bouyer branches: 1.1.2; 1.1.4;
file xenfunc.c was initially added on branch bouyer-xenamd64.
 1.1.4.2  18-Feb-2008  mjf Sync with HEAD.
 1.1.4.1  08-Dec-2007  mjf Sync with HEAD.
 1.1.2.2  21-Oct-2007  bouyer Factorise some Xen pmap code in x86_xpmap.c.
More xpmap_{ptom,mtop} -> xpmap_{ptom,mtop}_masked

The xenamd64 kernel is now good enough to complete a sysinst install from
xennet to xbd.
 1.1.2.1  17-Oct-2007  bouyer Prepare for xenamd64:
- kill xen/i386/identcpu.c, use i386/i386/identcpu.c instead (with a few
#ifndef XEN)
- move some files that can be shared between i386 and amd64 from
xen/i386 to xen/x86 (or to xen/xen for non-cpu-specific code)
- split assembly out of xen/include/hypervisor.h to xen/include/hypercalls.h
- use <xen/...> instead of <machine/...> for cpu-independant include files.

more work needed here, i386-specific files should got out of arch/xen to
arch/xeni386, and more code shared with arch/i386.
 1.2.16.3  23-Mar-2008  matt sync with HEAD
 1.2.16.2  09-Jan-2008  matt sync with HEAD
 1.2.16.1  22-Nov-2007  matt file xenfunc.c was added on branch matt-armv6 on 2008-01-09 01:50:17 +0000
 1.2.12.1  09-Jan-2008  bouyer Merge xen bits to i386/i386/gdt.c. Convert remaining uses of PTE_* macros to
pmap_pte_* macros/inlines.
Fix think-o in pmap.c for native i386.
 1.2.8.4  27-Feb-2008  yamt sync with head.
 1.2.8.3  21-Jan-2008  yamt sync with head
 1.2.8.2  07-Dec-2007  yamt sync with head
 1.2.8.1  22-Nov-2007  yamt file xenfunc.c was added on branch yamt-lazymbuf on 2007-12-07 17:27:19 +0000
 1.2.4.2  03-Dec-2007  ad Sync with HEAD.
 1.2.4.1  22-Nov-2007  ad file xenfunc.c was added on branch vmlocking on 2007-12-03 19:04:45 +0000
 1.2.2.2  27-Nov-2007  joerg Sync with HEAD. amd64 Xen support needs testing.
 1.2.2.1  22-Nov-2007  joerg file xenfunc.c was added on branch jmcneill-pm on 2007-11-27 19:36:24 +0000
 1.4.8.1  18-May-2008  yamt sync with head.
 1.4.6.1  02-Jun-2008  mjf Sync with HEAD.
 1.5.2.4  11-Aug-2010  yamt sync with head.
 1.5.2.3  11-Mar-2010  yamt sync with head
 1.5.2.2  19-Aug-2009  yamt sync with head.
 1.5.2.1  16-May-2008  yamt sync with head.
 1.6.2.1  23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.7.12.3  27-Aug-2011  jym Sync with HEAD. Most notably: uvm/pmap work done by rmind@, and MP Xen
work of cherry@.

No regression observed on suspend/restore.
 1.7.12.2  24-Oct-2010  jym Sync with HEAD
 1.7.12.1  01-Nov-2009  jym Sync with HEAD.
 1.9.2.2  17-Aug-2010  uebayasi Sync with HEAD.
 1.9.2.1  30-Apr-2010  uebayasi Sync with HEAD.
 1.10.2.1  05-Mar-2011  rmind sync with head
 1.11.6.3  20-Sep-2011  cherry Remove the "xpq lock", since we have per-cpu mmu queues now. This may need further testing. Also add some preliminary locking around queue-ops in the network backend driver
 1.11.6.2  31-Jul-2011  cherry grow MP support for i386. boots to single user
 1.11.6.1  03-Jun-2011  cherry Initial import of xen MP sources, with kernel and userspace tests.
- this is a source priview.
- boots to single user.
- spurious interrupt and pmap related panics are normal
 1.12.2.1  10-Nov-2011  yamt sync with head
 1.13.32.2  20-Mar-2017  pgoyette Sync with HEAD
 1.13.32.1  07-Jan-2017  pgoyette Sync with HEAD. (Note that most of these changes are simply $NetBSD$
tag issues.)
 1.13.28.3  28-Aug-2017  skrll Sync with HEAD
 1.13.28.2  05-Feb-2017  skrll Sync with HEAD
 1.13.28.1  05-Dec-2016  skrll Sync with HEAD
 1.13.10.1  03-Dec-2017  jdolecek update from HEAD
 1.15.2.1  21-Apr-2017  bouyer Sync with HEAD
 1.17.2.6  18-Jan-2019  pgoyette Synch with HEAD
 1.17.2.5  26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.17.2.4  20-Oct-2018  pgoyette Sync with head
 1.17.2.3  30-Sep-2018  pgoyette Ssync with HEAD
 1.17.2.2  28-Jul-2018  pgoyette Sync with HEAD
 1.17.2.1  25-Jun-2018  pgoyette Sync with HEAD
 1.18.2.1  10-Jun-2019  christos Sync with HEAD
 1.26.8.2  12-Apr-2020  bouyer kpreempt_disable() only for x86_64 (which calls pmap_changeprot_local)).
On i386 curcpu() is not valid yet and we don't need preemption disabled.
 1.26.8.1  11-Apr-2020  bouyer Move softint and preemtion-related functions out of x86/x86/intr.c to
its own file, x86/x86/x86_softintr.c
Add x86/x86/x86_softintr.c for native and XenPV
Make sure XenPV also check ci_ioending, which is used for softints.
Switch XenPV to fast softints and allow kernel preemption.
kpreempt_disable() before calling pmap_changeprot_local()
run xen_wallclock_time() and xen_global_systime_ns() at splshed() to
avoid being interrupted.

XXX amd64 lock stubs are racy for XPENDING

RSS XML Feed