branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8;
a simple performance monitor based profiler, inspired from linux oprofile.

KNF. No functional change.

tprof_lock is not a spin mutex. use mutex_{enter,exit}(). oops

branches: 1.21.2;
- Add support select(2)/poll(2) on /dev/tprof.
- Changed sampling buffer switching frequency (which is the frequency of tprof_worker()
calls and also the maximum block time of read(2) of /dev/tprof) from 1sec to 125ms.
This improve tprof top responsiveness.
- The maximum number of sampling buffers is now adjusted according to the number of CPUs.
Previously it was fixed at 100 and was insufficient if ncpu was greater than this.

The maximum number of samples per second per CPU is calculated by
"TPROF_MAX_SAMPLES_PER_BUF * (HZ of tprof_worker)".
Therefore, currently, 10000 * (1000/125) = 80000 maximum samplings per CPU.
The actual value will vary slightly from this due to tprof_worker and read(2) timing.
This value may need to be adjusted more in the future.

make sure error is initialized before we return it.

Improve tprof(4)

- Multiple events can now be handled simultaneously.
- Counters should be configured with TPROF_IOC_CONFIGURE_EVENT in advance,
instead of being configured at TPROF_IOC_START.
- The configured counters can be started and stopped repeatedly by
PROF_IOC_START/TPROF_IOC_STOP.
- The value of the performance counter can be obtained at any timing as a 64bit
value with TPROF_IOC_GETCOUNTS.
- Backend common parts are handled in tprof.c as much as possible, and functions
on the tprof_backend side have been reimplemented to be more primitive.
- The reset value of counter overflows for profiling can now be adjusted.
It is calculated by default from the CPU clock (speed of cycle counter) and
TPROF_HZ, but for some events the value may be too large to be sufficient for
profiling. The event counter can be specified as a ratio to the default or as
an absolute value when configuring the event counter.
- Due to overall changes, API and ABI have been changed. TPROF_VERSION and
TPROF_BACKEND_VERSION were updated.

don't call kpreempt_{disable,enable}() from an interrupt handler.

Fixed a problem in which the system would freeze if a high load (e.g., build.sh -j20)
was applied while running `tprof monitor -e LsNotHaltedCyc ...' on x86.

This almost eliminates the problem, but still is not enough. tprof_x86 uses NMI
interrupts, which are interrupted even in splhigh(), leaving the possibility of
being interrupted in the splhigh section of percpu_cpu_swap().

driver(9): devsw_detach never fails. Make it return void.

Prune a whole lotta dead branches as a result of this. (Some logic
calling this is also wrong for other reasons; devsw_detach is final
-- you should never have any reason to decide to roll it back. To be
cleaned up in subsequent commits...)

XXX kernel ABI change to devsw_detach signature requires bump

Trailing whitespace

tprof: Use percpu rather than a MAXCPUS-element array.

branches: 1.14.6; 1.14.14;
Revamp tprof.

Rewrite the Intel backend to use the generic PMC interface, which is
available on all Intel CPUs. Synchronize the AMD backend with the new
interface.

The kernel identifies the PMC interface, and gives its id to userland.
Userland then queries the events itself (via cpuid etc). These events
depend on the PMC interface.

The tprof utility is rewritten to allow the user to choose which event
to count (which was not possible until now, the event was hardcoded in
the backend). The command line format is based on usr.bin/pmc, eg:

tprof -e llc-misses:k -o output sleep 20

The man page is updated too, but the arguments will likely change soon
anyway so it doesn't matter a lot.

The tprof utility has three tables:

Intel Architectural Version 1
Intel Skylake/Kabylake
AMD Family 10h

A CPU can support a combination of tables. For example Kabylake has
Intel-Architectural-Version-1 and its own Intel-Kabylake table.

For now the Intel Skylake/Kabylake table contains only one event, just
to demonstrate that the combination of tables works. Tested on an
Intel Core i5 Kabylake.

The code for AMD Family 10h is taken from the code I had written for
usr.bin/pmc. I haven't tested it yet, but it's the same as pmc(1), so
I guess it works as-is.

The whole thing is written in such a way that (I think) it is not
complicated to add more CPU models, and more architectures (other than
x86).

branches: 1.13.8; 1.13.16; 1.13.18;
include "ioconf.h" to get the 'void <driver>attach(int count);' prototype.

branches: 1.12.4;
Add d_discard to all struct cdevsw instances I could find.

All have been set to "nodiscard"; some should get a real implementation.

branches: 1.11.2;
Change (mostly mechanically) every cdevsw/bdevsw I can find to use
designated initializers.

I have not built every extant kernel so I have probably broken at
least one build; however I've also found and fixed some wrong
cdevsw/bdevsw entries so even if so I think we come out ahead.

branches: 1.10.4; 1.10.14; 1.10.18;
for each samples, record and report cpuid and lwpid.

tprof_start: don't forget to restore refcount when failed to start backend.

tprof: record pid and userland events.

branches: 1.7.2; 1.7.4;
Keep condvar wmesg within 8-char limit

branches: 1.6.2; 1.6.4;
tprof_stop1: add an assertion.

fix breakage where db_regs_t != trapframe.
the problem pointed out by Martin Husemann on tech-kern@.

- adapt to MODULAR.
- some preparations to have more backends.
- add some comments.

branches: 1.3.2;
comment

branches: 1.2.8;
tprof_start: fix workqueue's IPL.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.14; 1.1.16; 1.1.18;
a simple performance monitor based profiler, inspired from linux oprofile.

Improve tprof(4)

- Multiple events can now be handled simultaneously.
- Counters should be configured with TPROF_IOC_CONFIGURE_EVENT in advance,
instead of being configured at TPROF_IOC_START.
- The configured counters can be started and stopped repeatedly by
PROF_IOC_START/TPROF_IOC_STOP.
- The value of the performance counter can be obtained at any timing as a 64bit
value with TPROF_IOC_GETCOUNTS.
- Backend common parts are handled in tprof.c as much as possible, and functions
on the tprof_backend side have been reimplemented to be more primitive.
- The reset value of counter overflows for profiling can now be adjusted.
It is calculated by default from the CPU clock (speed of cycle counter) and
TPROF_HZ, but for some events the value may be too large to be sufficient for
profiling. The event counter can be specified as a ratio to the default or as
an absolute value when configuring the event counter.
- Due to overall changes, API and ABI have been changed. TPROF_VERSION and
TPROF_BACKEND_VERSION were updated.

Revamp tprof.

Rewrite the Intel backend to use the generic PMC interface, which is
available on all Intel CPUs. Synchronize the AMD backend with the new
interface.

The kernel identifies the PMC interface, and gives its id to userland.
Userland then queries the events itself (via cpuid etc). These events
depend on the PMC interface.

The tprof utility is rewritten to allow the user to choose which event
to count (which was not possible until now, the event was hardcoded in
the backend). The command line format is based on usr.bin/pmc, eg:

tprof -e llc-misses:k -o output sleep 20

The man page is updated too, but the arguments will likely change soon
anyway so it doesn't matter a lot.

The tprof utility has three tables:

Intel Architectural Version 1
Intel Skylake/Kabylake
AMD Family 10h

A CPU can support a combination of tables. For example Kabylake has
Intel-Architectural-Version-1 and its own Intel-Kabylake table.

For now the Intel Skylake/Kabylake table contains only one event, just
to demonstrate that the combination of tables works. Tested on an
Intel Core i5 Kabylake.

The code for AMD Family 10h is taken from the code I had written for
usr.bin/pmc. I haven't tested it yet, but it's the same as pmc(1), so
I guess it works as-is.

The whole thing is written in such a way that (I think) it is not
complicated to add more CPU models, and more architectures (other than
x86).

branches: 1.5.54; 1.5.56;
tprof: record pid and userland events.

branches: 1.4.4; 1.4.6; 1.4.8;
comment

fix breakage where db_regs_t != trapframe.
the problem pointed out by Martin Husemann on tech-kern@.

- adapt to MODULAR.
- some preparations to have more backends.
- add some comments.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.18; 1.1.26; 1.1.32;
a simple performance monitor based profiler, inspired from linux oprofile.

KNF. No functional change.

Even if an overflow interrupt is occured for a counter outside tprof management,
the bit of onverflow status register must be cleared to prevent an interrupt storm.

branches: 1.11.2;
move ARMv7 PMC register definitions to armreg.h from tprof_armv7.c

Improve tprof(4)

- Multiple events can now be handled simultaneously.
- Counters should be configured with TPROF_IOC_CONFIGURE_EVENT in advance,
instead of being configured at TPROF_IOC_START.
- The configured counters can be started and stopped repeatedly by
PROF_IOC_START/TPROF_IOC_STOP.
- The value of the performance counter can be obtained at any timing as a 64bit
value with TPROF_IOC_GETCOUNTS.
- Backend common parts are handled in tprof.c as much as possible, and functions
on the tprof_backend side have been reimplemented to be more primitive.
- The reset value of counter overflows for profiling can now be adjusted.
It is calculated by default from the CPU clock (speed of cycle counter) and
TPROF_HZ, but for some events the value may be too large to be sufficient for
profiling. The event counter can be specified as a ratio to the default or as
an absolute value when configuring the event counter.
- Due to overall changes, API and ABI have been changed. TPROF_VERSION and
TPROF_BACKEND_VERSION were updated.

tprof_armv7 initializes on each CPUs, like tprof_armv8.

PMCR.E should not be disabled from tprof.

PMCR.E controls not only performance event counters but also the cycle
counter operation, and the cycle counter may be used for cpu_counter.
Similarly, the 31st bit in PMINTENCLR and PMCNTENCLR controls the cycle
counter, not performance event counters, and should not be modified.

Add support for Cortex-A9.

declare xc

Improve error handling.

Hypervisors can return a PMCR.N of 0.

Retire arm_[di]sb in favour of the isb() and dsb(sy) macro invocations.

0x%#x --> %#x for non-external codes.
Also, stop mixing up 0x%x and %#x in single files as far as possible.

branches: 1.2.2; 1.2.8; 1.2.12;
RW fields in performance monitor registers are reset to architecturally
UNKNOWN values. Initialize the PMU to a known state - all interrupts and
counters disabled, performance monitor disabled, and user access disabled.

Add tprof backend for ARMv7 performance monitors.

branches: 1.1.2; 1.1.8;
Add tprof backend for ARMv7 performance monitors.

KNF. No functional change.

Even if an overflow interrupt is occured for a counter outside tprof management,
the bit of onverflow status register must be cleared to prevent an interrupt storm.

branches: 1.18.2;
Improve tprof(4)

- Multiple events can now be handled simultaneously.
- Counters should be configured with TPROF_IOC_CONFIGURE_EVENT in advance,
instead of being configured at TPROF_IOC_START.
- The configured counters can be started and stopped repeatedly by
PROF_IOC_START/TPROF_IOC_STOP.
- The value of the performance counter can be obtained at any timing as a 64bit
value with TPROF_IOC_GETCOUNTS.
- Backend common parts are handled in tprof.c as much as possible, and functions
on the tprof_backend side have been reimplemented to be more primitive.
- The reset value of counter overflows for profiling can now be adjusted.
It is calculated by default from the CPU clock (speed of cycle counter) and
TPROF_HZ, but for some events the value may be too large to be sufficient for
profiling. The event counter can be specified as a ratio to the default or as
an absolute value when configuring the event counter.
- Due to overall changes, API and ABI have been changed. TPROF_VERSION and
TPROF_BACKEND_VERSION were updated.

PMCR.E should not be disabled from tprof.

PMCR.E controls not only performance event counters but also the cycle
counter operation, and the cycle counter may be used for cpu_counter.
Similarly, the 31st bit in PMINTENCLR and PMCNTENCLR controls the cycle
counter, not performance event counters, and should not be modified.

revert my previous commit.

since armv8_pmu_init is only called when it is reliably worked by ACPI or fdt,
there is no need for dynamic checks.

pointed out by jmcneill@, thanks

If the hardware does not support PMU, return an error instead of KASSERT.

tprof: armv8: Only attach to known PMU types.

fix the typo that martin spotted.

Add a comment and simplify the code ever so slightly.

Use the first (not second) event counter as there might only be one
available.

declare xc

Improve error handling.

Hypervisors can return a PMCR.N of 0.

Trailing whitespace

Make sure setup happens on all CPUs.

Retire arm_[di]sb in favour of the isb() and dsb(sy) macro invocations.

Enable the cycle counter when a CPU hatches and store an estimate of the
frequency in ci_data.cpu_cc_freq.

branches: 1.4.2; 1.4.8;
use PRI?64 instead of ll?

Spaces -> tabs

RW fields in performance monitor registers are reset to architecturally
UNKNOWN values. Initialize the PMU to a known state - all interrupts and
counters disabled, performance monitor disabled, and user access disabled.

Add tprof backend for ARMv8 performance monitors.

tprof: armv8: Only attach to known PMU types.

branches: 1.1.2; 1.1.8;
Add tprof backend for ARMv8 performance monitors.

Improve tprof(4)

- Multiple events can now be handled simultaneously.
- Counters should be configured with TPROF_IOC_CONFIGURE_EVENT in advance,
instead of being configured at TPROF_IOC_START.
- The configured counters can be started and stopped repeatedly by
PROF_IOC_START/TPROF_IOC_STOP.
- The value of the performance counter can be obtained at any timing as a 64bit
value with TPROF_IOC_GETCOUNTS.
- Backend common parts are handled in tprof.c as much as possible, and functions
on the tprof_backend side have been reimplemented to be more primitive.
- The reset value of counter overflows for profiling can now be adjusted.
It is calculated by default from the CPU clock (speed of cycle counter) and
TPROF_HZ, but for some events the value may be too large to be sufficient for
profiling. The event counter can be specified as a ratio to the default or as
an absolute value when configuring the event counter.
- Due to overall changes, API and ABI have been changed. TPROF_VERSION and
TPROF_BACKEND_VERSION were updated.

Revamp tprof.

Rewrite the Intel backend to use the generic PMC interface, which is
available on all Intel CPUs. Synchronize the AMD backend with the new
interface.

The kernel identifies the PMC interface, and gives its id to userland.
Userland then queries the events itself (via cpuid etc). These events
depend on the PMC interface.

The tprof utility is rewritten to allow the user to choose which event
to count (which was not possible until now, the event was hardcoded in
the backend). The command line format is based on usr.bin/pmc, eg:

tprof -e llc-misses:k -o output sleep 20

The man page is updated too, but the arguments will likely change soon
anyway so it doesn't matter a lot.

The tprof utility has three tables:

Intel Architectural Version 1
Intel Skylake/Kabylake
AMD Family 10h

A CPU can support a combination of tables. For example Kabylake has
Intel-Architectural-Version-1 and its own Intel-Kabylake table.

For now the Intel Skylake/Kabylake table contains only one event, just
to demonstrate that the combination of tables works. Tested on an
Intel Core i5 Kabylake.

The code for AMD Family 10h is taken from the code I had written for
usr.bin/pmc. I haven't tested it yet, but it's the same as pmc(1), so
I guess it works as-is.

The whole thing is written in such a way that (I think) it is not
complicated to add more CPU models, and more architectures (other than
x86).

branches: 1.3.54; 1.3.56;
for each samples, record and report cpuid and lwpid.

tprof: record pid and userland events.

branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8; 1.1.40; 1.1.46; 1.1.48;
a simple performance monitor based profiler, inspired from linux oprofile.

KNF. No functional change.

branches: 1.6.2;
Improve tprof(4)

- Multiple events can now be handled simultaneously.
- Counters should be configured with TPROF_IOC_CONFIGURE_EVENT in advance,
instead of being configured at TPROF_IOC_START.
- The configured counters can be started and stopped repeatedly by
PROF_IOC_START/TPROF_IOC_STOP.
- The value of the performance counter can be obtained at any timing as a 64bit
value with TPROF_IOC_GETCOUNTS.
- Backend common parts are handled in tprof.c as much as possible, and functions
on the tprof_backend side have been reimplemented to be more primitive.
- The reset value of counter overflows for profiling can now be adjusted.
It is calculated by default from the CPU clock (speed of cycle counter) and
TPROF_HZ, but for some events the value may be too large to be sufficient for
profiling. The event counter can be specified as a ratio to the default or as
an absolute value when configuring the event counter.
- Due to overall changes, API and ABI have been changed. TPROF_VERSION and
TPROF_BACKEND_VERSION were updated.

Add TPROF_IDENT_ARMV7_GENERIC

Define TPROF_IDENT_ARMV8_GENERIC

Revamp tprof.

Rewrite the Intel backend to use the generic PMC interface, which is
available on all Intel CPUs. Synchronize the AMD backend with the new
interface.

The kernel identifies the PMC interface, and gives its id to userland.
Userland then queries the events itself (via cpuid etc). These events
depend on the PMC interface.

The tprof utility is rewritten to allow the user to choose which event
to count (which was not possible until now, the event was hardcoded in
the backend). The command line format is based on usr.bin/pmc, eg:

tprof -e llc-misses:k -o output sleep 20

The man page is updated too, but the arguments will likely change soon
anyway so it doesn't matter a lot.

The tprof utility has three tables:

Intel Architectural Version 1
Intel Skylake/Kabylake
AMD Family 10h

A CPU can support a combination of tables. For example Kabylake has
Intel-Architectural-Version-1 and its own Intel-Kabylake table.

For now the Intel Skylake/Kabylake table contains only one event, just
to demonstrate that the combination of tables works. Tested on an
Intel Core i5 Kabylake.

The code for AMD Family 10h is taken from the code I had written for
usr.bin/pmc. I haven't tested it yet, but it's the same as pmc(1), so
I guess it works as-is.

The whole thing is written in such a way that (I think) it is not
complicated to add more CPU models, and more architectures (other than
x86).

branches: 1.2.4; 1.2.56; 1.2.58;
for each samples, record and report cpuid and lwpid.

branches: 1.1.2; 1.1.4;
tprof: record pid and userland events.

Improve tprof(4)

- Multiple events can now be handled simultaneously.
- Counters should be configured with TPROF_IOC_CONFIGURE_EVENT in advance,
instead of being configured at TPROF_IOC_START.
- The configured counters can be started and stopped repeatedly by
PROF_IOC_START/TPROF_IOC_STOP.
- The value of the performance counter can be obtained at any timing as a 64bit
value with TPROF_IOC_GETCOUNTS.
- Backend common parts are handled in tprof.c as much as possible, and functions
on the tprof_backend side have been reimplemented to be more primitive.
- The reset value of counter overflows for profiling can now be adjusted.
It is calculated by default from the CPU clock (speed of cycle counter) and
TPROF_HZ, but for some events the value may be too large to be sufficient for
profiling. The event counter can be specified as a ratio to the default or as
an absolute value when configuring the event counter.
- Due to overall changes, API and ABI have been changed. TPROF_VERSION and
TPROF_BACKEND_VERSION were updated.

branches: 1.1.2; 1.1.8;
Merge the tprof_pmi and tprof_amdpmi modules into a single tprof_x86
module.

KNF. No functional change.

branches: 1.7.2;
Add AMD family 19h (zen3 and zen4) support to tprof.

Improve tprof(4)

- Multiple events can now be handled simultaneously.
- Counters should be configured with TPROF_IOC_CONFIGURE_EVENT in advance,
instead of being configured at TPROF_IOC_START.
- The configured counters can be started and stopped repeatedly by
PROF_IOC_START/TPROF_IOC_STOP.
- The value of the performance counter can be obtained at any timing as a 64bit
value with TPROF_IOC_GETCOUNTS.
- Backend common parts are handled in tprof.c as much as possible, and functions
on the tprof_backend side have been reimplemented to be more primitive.
- The reset value of counter overflows for profiling can now be adjusted.
It is calculated by default from the CPU clock (speed of cycle counter) and
TPROF_HZ, but for some events the value may be too large to be sufficient for
profiling. The event counter can be specified as a ratio to the default or as
an absolute value when configuring the event counter.
- Due to overall changes, API and ABI have been changed. TPROF_VERSION and
TPROF_BACKEND_VERSION were updated.

Match Family 15h

branches: 1.4.2;
Fix compile error (s/LAPIC_PCINT/LAPIC_LVT_PCINT/)

branches: 1.3.2;
Add support for AMD Family 17h.

branches: 1.2.2;
Merge the tprof_pmi and tprof_amdpmi modules into a single tprof_x86
module.

Move
arch/x86/x86/tprof_pmi.c
arch/x86/x86/tprof_amdpmi.c
into
dev/tprof/tprof_x86_intel.c
dev/tprof/tprof_x86_amd.c

KNF. No functional change.

Test cpuid_level in tprof_intel_ncounters().

This function is called before tprof_intel_ident().

Obtain the number of general counters from CPUID 0xa.

branches: 1.5.2;
Improve tprof(4)

- Multiple events can now be handled simultaneously.
- Counters should be configured with TPROF_IOC_CONFIGURE_EVENT in advance,
instead of being configured at TPROF_IOC_START.
- The configured counters can be started and stopped repeatedly by
PROF_IOC_START/TPROF_IOC_STOP.
- The value of the performance counter can be obtained at any timing as a 64bit
value with TPROF_IOC_GETCOUNTS.
- Backend common parts are handled in tprof.c as much as possible, and functions
on the tprof_backend side have been reimplemented to be more primitive.
- The reset value of counter overflows for profiling can now be adjusted.
It is calculated by default from the CPU clock (speed of cycle counter) and
TPROF_HZ, but for some events the value may be too large to be sufficient for
profiling. The event counter can be specified as a ratio to the default or as
an absolute value when configuring the event counter.
- Due to overall changes, API and ABI have been changed. TPROF_VERSION and
TPROF_BACKEND_VERSION were updated.

Use CPUID_PERF_* macros defined in specialreg.h. No functional change.

branches: 1.3.2;
Fix compile error (s/LAPIC_PCINT/LAPIC_LVT_PCINT/)

branches: 1.2.2; 1.2.8;
Merge the tprof_pmi and tprof_amdpmi modules into a single tprof_x86
module.

Move
arch/x86/x86/tprof_pmi.c
arch/x86/x86/tprof_amdpmi.c
into
dev/tprof/tprof_x86_intel.c
dev/tprof/tprof_x86_amd.c