Home | History | Annotate | Download | only in kern
History log of /src/sys/kern/kern_tc.c
RevisionDateAuthorComments
 1.78  15-Jul-2025  andvar Fix various typos in comments.
 1.77  11-May-2024  andvar s/timetamp/timestamp/ in comment.
 1.76  30-Jul-2023  riastradh timecounter(9): Nix trailing whitespace.

No functional change intended.
 1.75  28-Jul-2023  riastradh timecounter(9): Link to phk's timecounter paper for reference.

No functional change intended.
 1.74  27-Jul-2023  riastradh kern: Restore non-atomic time_second symbol.

This is used by savecore(8), vmstat(8), and possibly other things.

XXX Should really teach dump and savecore(8) to use an intentionally
designed header, rather than relying on kvm(3) -- and make it work
for saving cores from other kernels so you don't have to boot the
same buggy kernel to get a core dump.
 1.73  17-Jul-2023  riastradh timecounter(9): Sprinkle membar_consumer around th->th_generation.

This code was apparently written under the misapprehension that
membar_producer on one side is good enough, but that doesn't
accomplish anything other than making the code unnecessarily obscure.

For semantics, you always always always need memory barriers to come
in pairs, with membar_consumer on the reading side if you want the
membar_producer to have on the writing side to have any useful
effect.

It is unfortunate that this might hurt performance, but that's an
unfortunate consequence of the design made without understanding
memory barriers, not an unfortunate consequence of the memory
barriers.

If it is really critical for the read side to avoid memory barriers,
then the write side needs to issue an IPI or xcall to effect memory
barriers -- at higher cost to the write side, of course.
 1.72  17-Jul-2023  riastradh timecounter(9): Use atomic_store_release/load_consume for timehands.

This probably fixes real bugs on Alpha and makes the synchronization
pattern clearer everywhere.
 1.71  17-Jul-2023  riastradh timecounter(9): Use seqlock for atomic snapshots of timebase.
 1.70  17-Jul-2023  riastradh Revert "timecounter(9): Use an ipi barrier on time_second/uptime rollover."

Evidently rump doesn't have ipi, so this won't work unless we have an
alternate approach for rump.
 1.69  17-Jul-2023  riastradh timecounter(9): No static; committed wrong version of patch.
 1.68  17-Jul-2023  riastradh timecounter(9): Limit scope of time__second/uptime.

Relevant only if __HAVE_ATOMIC64_LOADSTORE -- not updated otherwise.
 1.67  17-Jul-2023  riastradh timecounter(9): Use an ipi barrier on time_second/uptime rollover.

This way we only need __insn_barrier, not membar_consumer, on the
read side.
 1.66  17-Jul-2023  riastradh timecounter(9): Revert last -- timecounter_lock is already IPL_HIGH.
 1.65  17-Jul-2023  riastradh timecounter(9): Ward off interrupts during time_second/uptime update.

Only relevant during 32-bit wraparound, so the potential performance
impact of using splhigh here is negligible; indeed, we would have to
go out of our way to exercise this in tests before it will ever
happen in the next century.
 1.64  17-Jul-2023  riastradh timecounter(9): Fix thinko in previous.

Swapped the wrong variable in this mental macro expansion!
 1.63  17-Jul-2023  riastradh kern: Make time_second and time_uptime macros that work atomically.

These use atomic load on platforms with atomic 64-bit load, and
seqlocks on platforms without.

This has the unfortunate side effect of slightly reducing the real
times available on 32-bit platforms, from ending some time in the
year 584942417218 AD, available on 64-bit platforms, to ending some
time in the year 584942417355 AD. But during that slightly shorter
time, 32-bit platforms can avoid bugs arising from non-atomic access
to time_uptime and time_second.

Note: All platforms still have non-atomic access problems for
bintime, binuptime, nanotime, nanouptime, &c. This can be addressed
by putting a seqlock around timebasebin and possibly some other
variable -- to be done in a later change.

XXX kernel ABI change -- deleting symbols
 1.62  02-Jun-2021  riastradh timecounter(9): Zero the next pointer after tc_detach.

This reverts it to a state where tc_init works again.
 1.61  08-Apr-2021  simonb branches: 1.61.2; 1.61.6;
Sort includes.
 1.60  08-Apr-2021  simonb tc_init(): KASSERT that the timecounter hasn't previously been
initialised by checking the tc_next field. Prevents nastly endless
loops in the timecounter code.
 1.59  27-May-2020  rin branches: 1.59.4;
- remove #ifdef __FreeBSD__
- return (x) --> return x

No binary changes this time, trust me ;).
 1.58  27-May-2020  rin Oops, put back opt_ntp.h for options PPS_{SYNC,DEBUG}.
 1.57  27-May-2020  rin - kern_tc.c does not depend on "options NTP"
- u_int64_t --> uint64_t

No binary changes.
 1.56  27-May-2020  rin Tiny optimization; timecounters should not go bad ~ every ticktock.
 1.55  23-May-2020  ad Move proc_lock into the data segment. It was dynamically allocated because
at the time we had mutex_obj_alloc() but not __cacheline_aligned.
 1.54  02-Jan-2020  thorpej - Eliminate the global "boottime" variable, which was being accessed
without any synchronization against changes by e.g. clock_settime().
- Replace with new getbinboottime() / getnanoboottime() / getmicroboottime()
functions (naming mirrors that of other time access functions in kern_tc.c).
It returns the (maybe-converted) value of timebasebin, which also tracks
our estimate of when the system was booted (i.e. the legacy "boottime" was
redundant).

XXX There needs to be a lockless synchronization mechanism for reading
timebasebin, but this is a problem in kern_tc.c that pre-existed these
"boottime" changes. At least now the problem is centralized in one location.
 1.53  27-Dec-2019  msaitoh s/inital/initial/
 1.52  06-Oct-2019  uwe xc_barrier - convenience function to xc_broadcast() a nop.

Make the intent more clear and also avoid a bunch of (xcfunc_t)nullop
casts that gcc 8 -Wcast-function-type is not happy about.
 1.51  01-Jul-2018  riastradh Revert "Sprinkle cold conditionals to make tc_ticktock before inittimecounter."

Not needed if we're no longer doing tc_ticktock in printf.
 1.50  30-Jun-2018  riastradh Sprinkle cold conditionals to make tc_ticktock before inittimecounter.

Enables Xen to boot again.

XXX Maybe we should have a tc_ticktock_cold instead or something so we
don't have to reach this far into the call graph.
 1.49  13-Feb-2018  maxv branches: 1.49.2; 1.49.4;
Put time_second and time_uptime in different cache lines, probably saves
us some false sharing.
 1.48  02-Nov-2017  riastradh C99ify initialization of dummy_timecounter.
 1.47  09-Jun-2017  chs define a copy of getnanotime() named dtrace_getnanotime() so that
dtrace can know from the name that it should not allow setting
fbt probes on it. needed by dtrace.
 1.46  14-Sep-2013  martin branches: 1.46.6;
ifdef a variable like its use
 1.45  26-May-2013  kardel branches: 1.45.2;
Extend kernel PPS api with pps_ref_event().
pps_ref_event() allows capturing PPS time stamps
that are not generated at precisely 1Hz (e. g.
by reading a precision clock via callout()).

This extension allows clock drivers to supply PPS
time-stamps and drive the kernel NTP PLL
without the overhead of interrupt-handling and
-processing.
 1.44  13-Nov-2012  pooka kill some -Wunused-but-set-variable warnings
 1.43  21-Feb-2012  martin branches: 1.43.2;
Make time_second and time_uptime volatile, so the compiler knows they
may change during loops. Fixes the macppc build, which previously
died with:
src/sys/arch/macppc/dev/dbdma.c:62:6: error: assuming signed overflow does not occur when assuming that (X + c) < X is always false
 1.42  13-Apr-2010  pooka branches: 1.42.8; 1.42.12;
_KERNEL_OPT
 1.41  06-Feb-2010  uebayasi branches: 1.41.2; 1.41.4;
__inline -> inline
 1.40  14-Jun-2009  kardel Make PPS work with fast time counters (> 2GHz)
by making the pps count time stamp and the update
time stamp u_int64.
The time delta between two PPS events can now
be correctly calculated avoiding any unaccounted
for wraps with 32-bit counters.
 1.39  23-May-2009  ad - Add lwp_pctr(), get an LWP's preemption/ctxsw counter.
- Fix a preemption bug in CURCPU_IDLE_P() that can lead to a bogus
assertion failure on DEBUG kernels.
- Fix MP/preemption races with timecounter detachment.
 1.38  11-Jan-2009  christos branches: 1.38.2;
merge christos-time_t
 1.37  19-Jul-2008  kardel branches: 1.37.2;
Initialize timecounter_lock at IPL_HIGH. Prevents a "locking against myself"
when com.c (options PPS_SYNC) delivers a PPS time stamp during clock
interupt processing.
 1.36  11-May-2008  ad branches: 1.36.2; 1.36.4;
Fix locking botch. Pointed out by kardel@.
 1.35  08-May-2008  ad - Add tc_gonebad(): allows timecounter to be flagged as bad and removed at
the next clock tick.
- Remove time_lock, which is no longer required.
 1.34  28-Apr-2008  martin branches: 1.34.2;
Remove clause 3 and 4 from TNF licenses
 1.33  21-Apr-2008  ad branches: 1.33.2;
Make ntp, pmc, reboot, sysarch, time syscalls MPSAFE.
 1.32  10-Feb-2008  ad branches: 1.32.6; 1.32.8; 1.32.10;
Disable the event counters by default, since they create terrible cache
behaviour on MP systems.
 1.31  20-Jan-2008  joerg Now that __HAVE_TIMECOUNTER and __HAVE_GENERIC_TODR are invariants,
remove the conditionals and the code associated with the undef case.
 1.30  05-Jan-2008  ad Serialize calls to tc_windup().
 1.29  03-Jan-2008  dyoung Add tc_detach() for detaching a timecounter. It is good enough to
detach the AMD Geode SC1100 27 MHz clock, geodecntr(4), but there
may be synchronization problems on MP machines.
 1.28  15-Dec-2007  yamt fix a comment.
 1.27  30-Nov-2007  ad branches: 1.27.2; 1.27.6;
Use membar_*().
 1.26  23-Nov-2007  elad Kill another instance of KAUTH_GENERIC_ISSUSER.
 1.25  16-Nov-2007  ad Work around broken MD headers.
 1.24  15-Nov-2007  ad Kill the memory barriers: force users of the outgoing timehand to move
on quickly by also incrementing its generation number.
 1.23  15-Nov-2007  ad Oops; fix thinko.
 1.22  15-Nov-2007  ad Add a bit of locking around timecounter attachment / selection.
 1.21  21-Oct-2007  simonb branches: 1.21.2;
Adjust a comment. In NetBSD the time-related kernel function prototypes
are in <sys/timevar.h>, not <sys/time.h> like they are in FreeBSD.
 1.20  17-Aug-2007  ad branches: 1.20.2; 1.20.6;
Timecounters are lockless. Add conservative memory barriers to ensure that
loads and stores occur in the correct order.
 1.19  04-Mar-2007  christos branches: 1.19.2; 1.19.10; 1.19.14;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
 1.18  09-Feb-2007  ad branches: 1.18.2;
Merge newlock2 to head.
 1.17  04-Jan-2007  elad Consistent usage of KAUTH_GENERIC_ISSUSER.
 1.16  01-Nov-2006  yamt remove some __unused from function parameters.
 1.15  21-Oct-2006  kardel Avoid printf() the selected timecounter as it is called in the path of hardclock().
The information is available via sysctl anyway.
 1.14  12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.13  24-Sep-2006  kardel Reduce hardclock() code path length wrt/ multiply/divide by
re-calculating the th_scale value only when necessary.
As discussed with gdamore@. Tested at (sub-)usec level with
stratum 1 reference.
 1.12  10-Sep-2006  tsutsui branches: 1.12.2;
Fix splclock()/splx() mismatch in previous.
 1.11  07-Sep-2006  simonb branches: 1.11.2; 1.11.4;
Guard the timecounter manipulations including the call to tc_windup()
in tc_init() with splclock(). Fixes doubled-up "selected timecounter"
messages on some architectures, including pc532 and cobalt.

Fix suggested by Frank Kardel.
 1.10  03-Sep-2006  christos use c99 initializers, per gimpy's request
 1.9  02-Sep-2006  kardel move MAX_TCNAMELEN into timetc.h, use constant in tmp buffer allocation
 1.8  02-Sep-2006  christos add missing initializers
 1.7  06-Aug-2006  bjh21 Using humanize_number() on clock speeds does more harm than good, and
precious little of either. Go back to displaying them with all their digits.
 1.6  05-Aug-2006  bjh21 Clock frequencies tend to be big numbers -- use humanize_number() when
displaying them through printf.
 1.5  23-Jul-2006  ad Use the LWP cached credentials where sane.
 1.4  14-Jul-2006  kardel rename boottimebin to timebasebin as this
struct actually keeps the start of the UTC
time scale and not the boot time. the relationship
is: utc-time = up-time + timebase.
background: when doing an ACPI sleep the uptime
freezes and on wakeup the tc_setclock() leads to
a new timebasebin - this had no relationship with
a boottime as the structure was previously called.

discussed on tech-kern@
anomalies (moving boottime, uptime describing running time)
where discovered by Arnaud Lacombe.
 1.3  09-Jun-2006  kardel branches: 1.3.4;
re-order initialization sequence to have real counters available during autoconfig
 1.2  07-Jun-2006  kardel merge FreeBSD timecounters from branch simonb-timecounters
- struct timeval time is gone
time.tv_sec -> time_second
- struct timeval mono_time is gone
mono_time.tv_sec -> time_uptime
- access to time via
{get,}{micro,nano,bin}time()
get* versions are fast but less precise
- support NTP nanokernel implementation (NTP API 4)
- further reading:
Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf
NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html
 1.1  04-Feb-2006  simonb branches: 1.1.1;
Initial revision
 1.1.1.1  04-Feb-2006  simonb branches: 1.1.1.1.2; 1.1.1.1.4; 1.1.1.1.6; 1.1.1.1.12;
Import main FreeBSD timecounter support files as of 4th Feb 2006.
 1.1.1.1.12.1  19-Jun-2006  chap Sync with head.
 1.1.1.1.6.4  14-Sep-2006  yamt sync with head.
 1.1.1.1.6.3  03-Sep-2006  yamt sync with head.
 1.1.1.1.6.2  11-Aug-2006  yamt sync with head
 1.1.1.1.6.1  26-Jun-2006  yamt sync with head.
 1.1.1.1.4.2  18-Feb-2006  yamt sync with head.
 1.1.1.1.4.1  04-Feb-2006  yamt file kern_tc.c was added on branch yamt-uio_vmspace on 2006-02-18 15:39:18 +0000
 1.1.1.1.2.11  05-Jun-2006  kardel keep -current ABI until user level code needs to make use of extended features (suggested by drochner@)
 1.1.1.1.2.10  02-Jun-2006  kardel - track kauth change
 1.1.1.1.2.9  01-Jun-2006  kardel include evcnt.h - eventcounter are used
 1.1.1.1.2.8  24-Apr-2006  kardel allocate more space for each single counter information
 1.1.1.1.2.7  22-Apr-2006  kardel - fix setting of timecounter by sysctl by providing a valid length indication
when creating the kern.timecounter.hardware sysctl node
- compile calling of pps function of timecounter unconditionally as
MP TSC code can use that to derive the 1 second calibration drill
 1.1.1.1.2.6  28-Feb-2006  kardel sysctl variables kern.timecounter.{choice,hardware}
printf cleanup
 1.1.1.1.2.5  05-Feb-2006  simonb Clean up the naming of the timecounter event counters.
 1.1.1.1.2.4  04-Feb-2006  simonb Adapt for NetBSD, use evcnts instead of sysctls for counters,
KASSERT NetBSD/FreeBSD differences.
XXX - need to add convert FreeBSD sysctls to NetBSD sysctls.
 1.1.1.1.2.3  04-Feb-2006  simonb Add NetBSD RCS ids, comment out __FBSDID().
 1.1.1.1.2.2  04-Feb-2006  simonb Pull up from HEAD.
 1.1.1.1.2.1  04-Feb-2006  simonb file kern_tc.c was added on branch simonb-timecounters on 2006-02-04 08:14:13 +0000
 1.3.4.9  11-Feb-2008  yamt sync with head.
 1.3.4.8  21-Jan-2008  yamt sync with head
 1.3.4.7  07-Dec-2007  yamt sync with head
 1.3.4.6  27-Oct-2007  yamt sync with head.
 1.3.4.5  03-Sep-2007  yamt sync with head.
 1.3.4.4  26-Feb-2007  yamt sync with head.
 1.3.4.3  30-Dec-2006  yamt sync with head.
 1.3.4.2  21-Jun-2006  yamt sync with head.
 1.3.4.1  09-Jun-2006  yamt file kern_tc.c was added on branch yamt-lazymbuf on 2006-06-21 15:09:38 +0000
 1.11.4.2  09-Sep-2006  rpaulo sync with head
 1.11.4.1  07-Sep-2006  rpaulo file kern_tc.c was added on branch rpaulo-netinet-merge-pcb on 2006-09-09 02:57:16 +0000
 1.11.2.3  06-Feb-2007  ad Quieten noisy boot messages.
 1.11.2.2  12-Jan-2007  ad Sync with head.
 1.11.2.1  18-Nov-2006  ad Sync with head.
 1.12.2.2  10-Dec-2006  yamt sync with head.
 1.12.2.1  22-Oct-2006  yamt sync with head
 1.18.2.1  12-Mar-2007  rmind Sync with HEAD.
 1.19.14.5  03-Dec-2007  joerg Sync with HEAD.
 1.19.14.4  27-Nov-2007  joerg Sync with HEAD. amd64 Xen support needs testing.
 1.19.14.3  21-Nov-2007  joerg Sync with HEAD.
 1.19.14.2  26-Oct-2007  joerg Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.
 1.19.14.1  03-Sep-2007  jmcneill Sync with HEAD.
 1.19.10.1  03-Sep-2007  skrll Sync with HEAD.
 1.19.2.1  20-Aug-2007  ad Sync with HEAD.
 1.20.6.2  18-Nov-2007  bouyer Sync with HEAD
 1.20.6.1  25-Oct-2007  bouyer Sync with HEAD.
 1.20.2.3  23-Mar-2008  matt sync with HEAD
 1.20.2.2  09-Jan-2008  matt sync with HEAD
 1.20.2.1  06-Nov-2007  matt sync with HEAD
 1.21.2.4  18-Feb-2008  mjf Sync with HEAD.
 1.21.2.3  27-Dec-2007  mjf Sync with HEAD.
 1.21.2.2  08-Dec-2007  mjf Sync with HEAD.
 1.21.2.1  19-Nov-2007  mjf Sync with HEAD.
 1.27.6.3  23-Jan-2008  bouyer Sync with HEAD.
 1.27.6.2  08-Jan-2008  bouyer Sync with HEAD
 1.27.6.1  02-Jan-2008  bouyer Sync with HEAD
 1.27.2.1  26-Dec-2007  ad Sync with head.
 1.32.10.1  18-May-2008  yamt sync with head.
 1.32.8.3  30-Dec-2008  christos fix dev_t and time_t format.
 1.32.8.2  01-Nov-2008  christos Sync with head.
 1.32.8.1  29-Mar-2008  christos Welcome to the time_t=long long dev_t=uint64_t branch.
 1.32.6.3  17-Jan-2009  mjf Sync with HEAD.
 1.32.6.2  28-Sep-2008  mjf Sync with HEAD.
 1.32.6.1  02-Jun-2008  mjf Sync with HEAD.
 1.33.2.5  11-Aug-2010  yamt sync with head.
 1.33.2.4  11-Mar-2010  yamt sync with head
 1.33.2.3  20-Jun-2009  yamt sync with head
 1.33.2.2  04-May-2009  yamt sync with head.
 1.33.2.1  16-May-2008  yamt sync with head.
 1.34.2.2  18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.34.2.1  23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.36.4.1  19-Oct-2008  haad Sync with HEAD.
 1.36.2.1  28-Jul-2008  simonb Sync with head.
 1.37.2.1  19-Jan-2009  skrll Sync with HEAD.
 1.38.2.1  23-Jul-2009  jym Sync with HEAD.
 1.41.4.1  30-May-2010  rmind sync with head
 1.41.2.1  30-Apr-2010  uebayasi Sync with HEAD.
 1.42.12.1  24-Feb-2012  mrg sync to -current.
 1.42.8.3  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.42.8.2  16-Jan-2013  yamt sync with (a bit old) head
 1.42.8.1  17-Apr-2012  yamt sync with head
 1.43.2.4  03-Dec-2017  jdolecek update from HEAD
 1.43.2.3  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.43.2.2  23-Jun-2013  tls resync from head
 1.43.2.1  20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.45.2.1  18-May-2014  rmind sync with head
 1.46.6.1  28-Aug-2017  skrll Sync with HEAD
 1.49.4.3  13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.49.4.2  08-Apr-2020  martin Merge changes from current as of 20200406
 1.49.4.1  10-Jun-2019  christos Sync with HEAD
 1.49.2.1  28-Jul-2018  pgoyette Sync with HEAD
 1.59.4.1  17-Apr-2021  thorpej Sync with HEAD.
 1.61.6.1  06-Jun-2021  cjep sync with head
 1.61.2.1  17-Jun-2021  thorpej Sync w/ HEAD.

RSS XML Feed