Home | History | Annotate | only in /src/lib/libpthread/arch/x86_64
History log of /src/lib/libpthread/arch/x86_64
RevisionDateAuthorComments
 1.12 16-May-2009  ad Remove unused code that's confusing when using cscope/opengrok.
 1.11 28-Apr-2008  martin branches: 1.11.8;
Remove clause 3 and 4 from TNF licenses
 1.10 10-Feb-2008  ad branches: 1.10.4;
- Remove libpthread's atomic ops.
- Remove the old spinlock-based mutex and rwlock implementations.
- Use the atomic ops from libc.
 1.9 13-Nov-2007  ad Mutexes:

- Play scrooge again and chop more cycles off acquire/release.
- Spin while the lock holder is running on another CPU (adaptive mutexes).
- Do non-atomic release.

Threadreg:

- Add the necessary hooks to use a thread register.
- Add the code for i386, using %gs.
- Leave i386 code disabled until xen and COMPAT_NETBSD32 have the changes.
 1.8 13-Nov-2007  ad For PR bin/37347:

- Override __libc_thr_init() instead of using our own constructor.
- Add pthread__getenv() and use instead of getenv(). This is used before
we are up and running and unfortunatley getenv() takes locks.

Other changes:

- Cache the spinlock vectors in pthread__st. Internal spinlock operations
now take 1 function call instead of 3 (i386).
- Use pthread__self() internally, not pthread_self().
- Use __attribute__ ((visibility("hidden"))) in some places.
- Kill PTHREAD_MAIN_DEBUG.
 1.7 11-Sep-2007  ad Fix a dodgy bit of assembly.
 1.6 07-Sep-2007  ad - Don't take the mutex's spinlock (ptr_interlock) in pthread_cond_wait().
Instead, make the deferred wakeup list a per-thread array and pass down
the lwpid_t's that way.

- In pthread_cond_wait(), take the mutex before dealing with early wakeup.
In this way there should never be contention on the CV's spinlock if
the app follows POSIX rules (there should only be contention on the
user-provided mutex).

- Add a port of the kernel's rwlocks. The rwlock's spinlock is only taken if
there is contention. This is enabled where atomic ops are available. Right
now that is only i386 and amd64 because I don't have other hardware to
test with. It's trivial to add stubs for other architectures as long as
they have compare-and-swap. When we have proper atomic ops the old rwlock
code can be removed.

- Add a new mutex implementation that's similar to the kernel's mutexes, but
uses compare-and-swap to maintain the waiters list, so no spinlocks are
involved. Same caveats apply as for the rwlocks.
 1.5 07-Sep-2007  ad Add: pthread__atomic_cas_ptr, pthread__atomic_swap_ptr, pthread__membar_full
This is a stopgap until the thorpej-atomic branch is complete.
 1.4 21-Oct-2004  fvdl branches: 1.4.14; 1.4.18;
Fix thread context switching to take the stack ABI into account.
From Wolfgang Solfrank.
 1.3 08-Nov-2003  fvdl branches: 1.3.2;
Restore %rax correctly during a full context restore (oops).
 1.2 26-Jul-2003  salo netbsd.org->NetBSD.org
 1.1 30-Jan-2003  fvdl libpthread support for x86_64.
 1.3.2.1 12-Nov-2004  jmc Pullup rev 1.4 (requested by fvdl in ticket #956)

Fix thread context switching to take the stack ABI into account.
 1.4.18.3 23-Mar-2008  matt sync with HEAD
 1.4.18.2 09-Jan-2008  matt sync with HEAD
 1.4.18.1 06-Nov-2007  matt sync with HEAD
 1.4.14.1 10-Sep-2007  skrll Sync with HEAD.
 1.10.4.1 18-May-2008  yamt sync with head.
 1.11.8.2 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.11.8.1 28-Apr-2008  martin file _context_u.S was added on branch christos-time_t on 2008-04-28 20:23:03 +0000
 1.7 16-May-2009  ad Remove unused code that's confusing when using cscope/opengrok.
 1.6 28-Apr-2008  martin branches: 1.6.8;
Remove clause 3 and 4 from TNF licenses
 1.5 02-Mar-2007  ad branches: 1.5.12;
Remove the PTHREAD_SA option. If M:N threads is reimplemented it's
better off done with a seperate library.
 1.4 07-Sep-2003  cl Remove possible race condition in upcall recycling.
 1.3 17-Jul-2003  nathanw Adapt to structure name changes.
 1.2 26-Jun-2003  nathanw Remove PT_SLEEPUC and add PT_TRAPUC.
 1.1 30-Jan-2003  fvdl libpthread support for x86_64.
 1.5.12.1 18-May-2008  yamt sync with head.
 1.6.8.2 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.6.8.1 28-Apr-2008  martin file genassym.cf was added on branch christos-time_t on 2008-04-28 20:23:03 +0000
 1.13 25-May-2023  riastradh libpthread: New pthread__smt_wait to put CPU in low power for spin.

This is now distinct from pthread__smt_pause, which is for spin lock
backoff with no paired wakeup.

On Arm, there is a single-bit event register per CPU, and there are two
instructions to manage it:

- wfe, wait for event -- if event register is clear, enter low power
mode and wait until event register is set; then exit low power mode
and clear event register

- sev, signal event -- sets event register on all CPUs (other
circumstances like interrupts also set the event register and cause
wfe to wake)

These can be used to reduce the power consumption of spinning for a
lock, but only if they are actually paired -- if there's no sev, wfe
might hang indefinitely. Currently only pthread_spin(3) actually
pairs them; the other lock primitives (internal lock, mutex, rwlock)
do not -- they have spin lock backoff loops, but no corresponding
wakeup to cancel a wfe.

It may be worthwhile to teach the other lock primitives to pair
wfe/sev, but that requires some performance measurement to verify
it's actually worthwhile. So for now, we just make sure not to use
wfe when there's no sev, and keep everything else the same -- this
should fix severe performance degredation in libpthread on Arm
without hurting anything else.

No change in the generated code on amd64 and i386. No change in the
generated code for pthread_spin.c on arm and aarch64 -- changes only
the generated code for pthread_lock.c, pthread_mutex.c, and
pthread_rwlock.c, as intended.

PR port-arm/57437

XXX pullup-10
 1.12 25-Jan-2011  christos branches: 1.12.36; 1.12.46; 1.12.54;
make pthread__sp unsigned long.
 1.11 16-May-2009  ad branches: 1.11.2;
Remove unused code that's confusing when using cscope/opengrok.
 1.10 28-Apr-2008  martin branches: 1.10.8;
Remove clause 3 and 4 from TNF licenses
 1.9 22-Mar-2008  ad branches: 1.9.2;
Cheat and add inlines for _atomic_cas_ptr() to work around gcc emitting
unneeded PIC stuff in mutex_lock() and mutex_unlock(), when a thread
register is used.
 1.8 10-Feb-2008  ad - Remove libpthread's atomic ops.
- Remove the old spinlock-based mutex and rwlock implementations.
- Use the atomic ops from libc.
 1.7 13-Nov-2007  ad Mutexes:

- Play scrooge again and chop more cycles off acquire/release.
- Spin while the lock holder is running on another CPU (adaptive mutexes).
- Do non-atomic release.

Threadreg:

- Add the necessary hooks to use a thread register.
- Add the code for i386, using %gs.
- Leave i386 code disabled until xen and COMPAT_NETBSD32 have the changes.
 1.6 24-Sep-2007  skrll Resurrect the function pointers for lock operations and allow each
architecture to provide asm versions of the RAS operations.

We do this because relying on the compiler to get the RAS right is not
sensible. (It gets alpha wrong and hppa is suboptimal)

Provide asm RAS ops for hppa.

(A slightly different version) reviewed by Andrew Doran.
 1.5 07-Sep-2007  ad Add: pthread__atomic_cas_ptr, pthread__atomic_swap_ptr, pthread__membar_full
This is a stopgap until the thorpej-atomic branch is complete.
 1.4 24-Dec-2005  perry branches: 1.4.8; 1.4.12;
Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.
 1.3 26-Jul-2003  salo netbsd.org->NetBSD.org
 1.2 17-Apr-2003  fvdl Catch up with changed struct reg (same as gregs in mcontext now).
 1.1 30-Jan-2003  fvdl libpthread support for x86_64.
 1.4.12.3 23-Mar-2008  matt sync with HEAD
 1.4.12.2 09-Jan-2008  matt sync with HEAD
 1.4.12.1 06-Nov-2007  matt sync with HEAD
 1.4.8.1 10-Sep-2007  skrll Sync with HEAD.
 1.9.2.1 18-May-2008  yamt sync with head.
 1.10.8.2 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.10.8.1 28-Apr-2008  martin file pthread_md.h was added on branch christos-time_t on 2008-04-28 20:23:03 +0000
 1.11.2.1 08-Feb-2011  bouyer Sync with HEAD
 1.12.54.1 01-Aug-2023  martin Pull up following revision(s) (requested by riastradh in ticket #296):

lib/libpthread/arch/x86_64/pthread_md.h: revision 1.13
lib/libpthread/pthread_int.h: revision 1.110
lib/libpthread/pthread_int.h: revision 1.111
lib/libpthread/arch/i386/pthread_md.h: revision 1.21
lib/libpthread/arch/arm/pthread_md.h: revision 1.12
lib/libpthread/arch/arm/pthread_md.h: revision 1.13
lib/libpthread/pthread_spin.c: revision 1.11
lib/libpthread/arch/aarch64/pthread_md.h: revision 1.2

libpthread: Use __nothing, not /* nothing */, for empty macros.

No functional change intended -- just safer to do it this way in case
the macros are used in if branches or comma expressions.

PR port-arm/57437 (pthread__smt_pause/wake issue)

libpthread: New pthread__smt_wait to put CPU in low power for spin.

This is now distinct from pthread__smt_pause, which is for spin lock
backoff with no paired wakeup.

On Arm, there is a single-bit event register per CPU, and there are two
instructions to manage it:
- wfe, wait for event -- if event register is clear, enter low power
mode and wait until event register is set; then exit low power mode
and clear event register
- sev, signal event -- sets event register on all CPUs (other
circumstances like interrupts also set the event register and cause
wfe to wake)

These can be used to reduce the power consumption of spinning for a
lock, but only if they are actually paired -- if there's no sev, wfe
might hang indefinitely. Currently only pthread_spin(3) actually
pairs them; the other lock primitives (internal lock, mutex, rwlock)
do not -- they have spin lock backoff loops, but no corresponding
wakeup to cancel a wfe.

It may be worthwhile to teach the other lock primitives to pair
wfe/sev, but that requires some performance measurement to verify
it's actually worthwhile. So for now, we just make sure not to use
wfe when there's no sev, and keep everything else the same -- this
should fix severe performance degredation in libpthread on Arm
without hurting anything else.

No change in the generated code on amd64 and i386. No change in the
generated code for pthread_spin.c on arm and aarch64 -- changes only
the generated code for pthread_lock.c, pthread_mutex.c, and
pthread_rwlock.c, as intended.

PR port-arm/57437
 1.12.46.1 04-Aug-2023  martin Pull up following revision(s) (requested by riastradh in ticket #1700):

lib/libpthread/arch/x86_64/pthread_md.h: revision 1.13
lib/libpthread/pthread_int.h: revision 1.110
lib/libpthread/pthread_int.h: revision 1.111
lib/libpthread/arch/i386/pthread_md.h: revision 1.21
lib/libpthread/arch/arm/pthread_md.h: revision 1.12
lib/libpthread/arch/arm/pthread_md.h: revision 1.13
lib/libpthread/pthread_spin.c: revision 1.11
lib/libpthread/arch/aarch64/pthread_md.h: revision 1.2

libpthread: Use __nothing, not /* nothing */, for empty macros.

No functional change intended -- just safer to do it this way in case
the macros are used in if branches or comma expressions.
PR port-arm/57437 (pthread__smt_pause/wake issue)

libpthread: New pthread__smt_wait to put CPU in low power for spin.

This is now distinct from pthread__smt_pause, which is for spin lock
backoff with no paired wakeup.

On Arm, there is a single-bit event register per CPU, and there are two
instructions to manage it:
- wfe, wait for event -- if event register is clear, enter low power
mode and wait until event register is set; then exit low power mode
and clear event register
- sev, signal event -- sets event register on all CPUs (other
circumstances like interrupts also set the event register and cause
wfe to wake)

These can be used to reduce the power consumption of spinning for a
lock, but only if they are actually paired -- if there's no sev, wfe
might hang indefinitely. Currently only pthread_spin(3) actually
pairs them; the other lock primitives (internal lock, mutex, rwlock)
do not -- they have spin lock backoff loops, but no corresponding
wakeup to cancel a wfe.

It may be worthwhile to teach the other lock primitives to pair
wfe/sev, but that requires some performance measurement to verify
it's actually worthwhile. So for now, we just make sure not to use
wfe when there's no sev, and keep everything else the same -- this
should fix severe performance degredation in libpthread on Arm
without hurting anything else.

No change in the generated code on amd64 and i386. No change in the
generated code for pthread_spin.c on arm and aarch64 -- changes only
the generated code for pthread_lock.c, pthread_mutex.c, and
pthread_rwlock.c, as intended.
PR port-arm/57437
 1.12.36.1 04-Aug-2023  martin Pull up following revision(s) (requested by riastradh in ticket #1878):

lib/libpthread/arch/x86_64/pthread_md.h: revision 1.13
lib/libpthread/pthread_int.h: revision 1.110
lib/libpthread/pthread_int.h: revision 1.111
lib/libpthread/arch/i386/pthread_md.h: revision 1.21
lib/libpthread/arch/arm/pthread_md.h: revision 1.12
lib/libpthread/arch/arm/pthread_md.h: revision 1.13
lib/libpthread/pthread_spin.c: revision 1.11
lib/libpthread/arch/aarch64/pthread_md.h: revision 1.2

libpthread: Use __nothing, not /* nothing */, for empty macros.

No functional change intended -- just safer to do it this way in case
the macros are used in if branches or comma expressions.
PR port-arm/57437 (pthread__smt_pause/wake issue)

libpthread: New pthread__smt_wait to put CPU in low power for spin.

This is now distinct from pthread__smt_pause, which is for spin lock
backoff with no paired wakeup.

On Arm, there is a single-bit event register per CPU, and there are two
instructions to manage it:
- wfe, wait for event -- if event register is clear, enter low power
mode and wait until event register is set; then exit low power mode
and clear event register
- sev, signal event -- sets event register on all CPUs (other
circumstances like interrupts also set the event register and cause
wfe to wake)

These can be used to reduce the power consumption of spinning for a
lock, but only if they are actually paired -- if there's no sev, wfe
might hang indefinitely. Currently only pthread_spin(3) actually
pairs them; the other lock primitives (internal lock, mutex, rwlock)
do not -- they have spin lock backoff loops, but no corresponding
wakeup to cancel a wfe.

It may be worthwhile to teach the other lock primitives to pair
wfe/sev, but that requires some performance measurement to verify
it's actually worthwhile. So for now, we just make sure not to use
wfe when there's no sev, and keep everything else the same -- this
should fix severe performance degredation in libpthread on Arm
without hurting anything else.

No change in the generated code on amd64 and i386. No change in the
generated code for pthread_spin.c on arm and aarch64 -- changes only
the generated code for pthread_lock.c, pthread_mutex.c, and
pthread_rwlock.c, as intended.
PR port-arm/57437
 1.13 02-Mar-2007  ad Remove the PTHREAD_SA option. If M:N threads is reimplemented it's
better off done with a seperate library.
 1.12 09-Jan-2006  skrll I was a bit over zealous with my last change so revert the
locked_return_point change.

Loading the instruction at locked_return_point as a return value didn't
work so well.

Thanks to uwe, cube, and dsl.
 1.11 04-Jan-2006  skrll A couple of fixes to make libpthread really shared, i.e. not have text re-
locations:

- Don't declare pthread__switch_away global
- Do the PIC dance for pthread__switch_return_point and
pthread__locked_switch. Ideally these (and other) symbols would
be hidden.

Thanks to uwe@, dyoung@ and elad@ for help.

XXX sh3 is still to be done.
XXX vax does strange things.
 1.10 23-Apr-2004  simonb branches: 1.10.2;
s/the the/the/ (only in sources that aren't regularly imported from
elsewhere).
 1.9 08-Nov-2003  fvdl Make register usage more consistent, also in comparison with the i386
version, for easier maintenance.
 1.8 20-Oct-2003  fvdl Correct and simplify computing the return point; just use PC-relative
addressing.
 1.7 19-Oct-2003  fvdl Make sure the stack stays aligned.
 1.6 07-Sep-2003  cl Remove possible race condition in upcall recycling.
 1.5 26-Jul-2003  salo netbsd.org->NetBSD.org
 1.4 26-Jun-2003  nathanw Adapt to pt_trapuc: change STACK_SWITCH to check for a value in pt_trapuc
and use it preferentially to a value in pt_uc, clearing it once on the new
stack. Move stores into pt_uc back to before the stack switch; storing
after the stack switch opened a one-instruction race condition where an upcall
that had just started a chain could be preempted again, and would bomb when
restarted due to its pt_uc not yet having been updated. Now that pt_trapuc
is what the upcall code writes to, it is safe to store to pt_uc before
switching stacks.

Remove obsolete pt_sleepuc code.
 1.3 12-Jun-2003  fvdl Apply Nathan's switch-away fix.
 1.2 10-Feb-2003  fvdl Continue at the plain switch return point in pthread__switch, not the
locked one, in the !PIC case. From Tor Egge via Havard Eidnes.
 1.1 30-Jan-2003  fvdl libpthread support for x86_64.
 1.10.2.2 11-Jan-2006  tron Pull up following revision(s) (requested by skrll in ticket #1115):
lib/libpthread/arch/x86_64/pthread_switch.S: revision 1.12
I was a bit over zealous with my last change so revert the
locked_return_point change.
Loading the instruction at locked_return_point as a return value didn't
work so well.
Thanks to uwe, cube, and dsl.
 1.10.2.1 08-Jan-2006  riz Pull up following revision(s) (requested by skrll in ticket #1093):
lib/libpthread/arch/sparc/pthread_switch.S: revision 1.8
lib/libpthread/arch/x86_64/pthread_switch.S: revision 1.11
lib/libpthread/arch/sparc64/pthread_switch.S: revision 1.9
lib/libpthread/arch/i386/pthread_switch.S: revision 1.9
A couple of fixes to make libpthread really shared, i.e. not have text re-
locations:
- Don't declare pthread__switch_away global
- Do the PIC dance for pthread__switch_return_point and
pthread__locked_switch. Ideally these (and other) symbols would
be hidden.
Thanks to uwe@, dyoung@ and elad@ for help.
XXX sh3 is still to be done.
XXX vax does strange things.

RSS XML Feed