Home | History | Annotate | Download | only in ata
History log of /src/sys/dev/ata/ata.c
RevisionDateAuthorComments
 1.171  17-Feb-2025  jakllsch Restore ATA DMA mode downgrade support everywhere; it's a necessary part
of any system supporting (parallel) ATA DMA. There is hardware out there,
including cobalt, macppc, and sparc64 where this functionality is
necessary to avoid non-functional disks, either in as-shipped hardware
configurations or with add-in cards, or perhaps just with compromised
IDE/PATA cables.

Should address:
PR 58767
PR 59023
PR 59078

If anyone really insists on not having this support they can now turn it
off themselves with `options ATA_NO_DOWNGRADE_MODE`
 1.170  17-Feb-2025  jakllsch Fix aprint_error_dev() abuse
 1.169  31-May-2022  andvar fix various typos in comments, documentation and messages.
 1.168  28-May-2022  andvar fix various typos in comments.
 1.167  14-Mar-2022  perseant Avoid an unaccounted extra channel freeze, if a reset is requested
more than once before the thread services the request. Closes PR#56745.
 1.166  23-Feb-2022  andvar fix various typos in comments, mainly immediatly/immediately/,
as well shared and recently fixed typos in OpenBSD code by Jonathan Grey.
 1.165  12-Nov-2021  skrll Trailing whitespace
 1.164  05-Oct-2021  rin PR kern/56403

Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.

(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.

Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().

(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).

"Go ahead" by jdolecek@.
 1.163  29-Aug-2021  rin Output missing '\n' for capability list when DMA support is not compiled in.
 1.162  07-Aug-2021  thorpej Merge thorpej-cfargs2.
 1.161  24-Apr-2021  thorpej branches: 1.161.8;
Merge thorpej-cfargs branch:

Simplify and make extensible the config_search() / config_found() /
config_attach() interfaces: rather than having different variants for
which arguments you want pass along, just have a single call that
takes a variadic list of tag-value arguments.

Adjust all call sites:
- Simplify wherever possible; don't pass along arguments that aren't
actually needed.
- Don't be explicit about what interface attribute is attaching if
the device only has one. (More simplification.)
- Add a config_probe() function to be used in indirect configuiration
situations, making is visibly easier to see when indirect config is
in play, and allowing for future change in semantics. (As of now,
this is just a wrapper around config_match(), but that is an
implementation detail.)

Remove unnecessary or redundant interface attributes where they're not
needed.

There are currently 5 "cfargs" defined:
- CFARG_SUBMATCH (submatch function for direct config)
- CFARG_SEARCH (search function for indirect config)
- CFARG_IATTR (interface attribte)
- CFARG_LOCATORS (locators array)
- CFARG_DEVHANDLE (devhandle_t - wraps OFW, ACPI, etc. handles)

...and a sentinel value CFARG_EOL.

Add some extra sanity checking to ensure that interface attributes
aren't ambiguous.

Use CFARG_DEVHANDLE in MI FDT, OFW, and ACPI code, and macppc and shark
ports to associate those device handles with device_t instance. This
will trickle trough to more places over time (need back-end for pre-OFW
Sun OBP; any others?).
 1.160  03-Oct-2020  riastradh branches: 1.160.4;
autoconf: Blame devices holding up boot with config_pending.

Blame message requires `boot -x' (AB_DEBUG).

Fix ata so it doesn't mismatch config_pending_incr/decr devices.
 1.159  25-May-2020  jdolecek disable downgrade of ATA mode from DMA, as generally not relevant
any more - while it has been instrumental to inadvertedly discover
driver bugs in PIO mode under QEMU recently, generally the switch
more hurts than helps, so now only warn when DMA errors happen

code kept under ATA_DOWNGRADE_MODE ifdef, disabled by default
 1.158  25-May-2020  jdolecek make ata_downgrade_mode() static, it's not used anywhere else
 1.157  02-May-2020  thorpej Back out changes to use a threadpool for now; it's causing trouble
for some folks on Thinkpads.
 1.156  25-Apr-2020  thorpej Rather than creating a kthread-per-channel, use a threadpool and a
threadpool-job-per-channel for the in-thread-context work that needs
to be done (which is rare).

On one of my test systems, this results in the total number of LWPs
after multi-user boot dropping from 116 to 78.
 1.155  13-Apr-2020  jdolecek fix use-after-free for ata xfer on bio submission found by KASAN

driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself

PR kern/55169 by Nick Hudson
 1.154  04-Apr-2020  jdolecek branches: 1.154.2;
fix deadlock in wdcwait() when xfer timeout happens while the atabus
thread sleeps in wdcwait() - check current lwp rather than relying
on global ATACH_TH_RUN channel flag

should fix the hang part of the problem reported in
http://mail-index.netbsd.org/netbsd-users/2020/03/12/msg024249.html

thanks to Paul Ripke for providing extensive debugging info
 1.153  21-Oct-2019  christos chuq does not like insomniac allocations so unlock-alloc-lock instead.
 1.152  21-Oct-2019  christos Fix assert_sleepable() panic by allocating with NOSLEEP. The alternative is
to unlock and relock the channel, but seems more dangerous to do so.
 1.151  15-Oct-2019  chs convert more KM_NOSLEEP to KM_SLEEP and remove code to handle failures.
 1.150  21-Aug-2019  msaitoh Fix typo (s/controler/controller/).
 1.149  25-May-2019  christos branches: 1.149.2;
use PR_ZERO
 1.148  06-Apr-2019  uwe Make sure the aprint_verbose chain is terminated with a newline.
Check sep instead of printed, b/c the latter is reset several times.
 1.147  11-Dec-2018  jdolecek in atabus_detach(), shutdown the channel thread only after all the
children successfully detach; this is especially important for
atapibus, which needs the thread to execute commands

fixes hang on poweroff with root on ATAPI cd(4), reported by Christoph Badura
 1.146  12-Nov-2018  jdolecek hold channel lock during whole ata_dmaerr()/ata_downgrade_mode() -
according to code inspection this is safe, none of the set_modes
hooks execute anything which would be taking the lock

adresses PR kern/53714 by Andreas Gustafsson
 1.145  24-Oct-2018  jdolecek ata_kill_active() needs to drop channel drop before calling kill_xfer() hook,
otherwise code tries to lock against itself when deactivating the xfer
 1.144  24-Oct-2018  jdolecek add further note about FIS-based switching to atastart()
 1.143  24-Oct-2018  jdolecek detach the controller itself on shutdown; adjust to not detach already
detached atabus/channel
 1.142  22-Oct-2018  jdolecek Merge jdolecek-ncqfixes branch

- ata_xfer's are dynamicall allocated as needed using a pool, no longer
limited to number of possible openings supported by controller; dump
and recovery paths use dedicated pre-allocated storage
- moved callouts and condvars from ata_xfer to queue or channel, so that
ata_xfer does not need special initialization
- slot allocation now done when xfer is being activated, uncoupled
from memory allocation; active slots are no longer tracked by controller
code
- channel and drive reset is done always via the atabus thread, and
now executes with channel locked the whole time
- NCQ recovery moved to shared function, and run via the thread also
- added some workarounds for buggy error recovery AHCI emulation in QEMU
and Parallels

designed to primarily fix kern/52614, but might also help with kern/47041
and kern/53183
 1.141  28-Oct-2017  riastradh branches: 1.141.2; 1.141.4; 1.141.6;
Kill some more extern struct cfdriver declarations.

Down with externs in .c!
 1.140  20-Oct-2017  jdolecek move ata_queue_alloc(1) and ata_queue_free() calls to ata_channel_init()
and ata_channel_destroy() respectively, to make attachment code simpler,
and to make it easier to spot special queue manipulation like cmdide(4)

on topic of PR kern/52606
 1.139  19-Oct-2017  jdolecek more detailed debug info; also sync DEBUG_* values in wd.c with ata.c
 1.138  18-Oct-2017  jdolecek add newlines to the debug messages
 1.137  15-Oct-2017  jdolecek move the extra thaw for scheduled ata_reset_channel() to the function itself,
so it's done regardless if the actual reset is run from thread context
or e.g. call with AT_POLL; fixes a hang after xfer failure and dma downgrade

add some debugging printfs, so freeze/thaw issues are easier to track
 1.136  10-Oct-2017  jdolecek split off functions used by 'wd* at umass?' into separate file, unfortunately
the previous approach with NATABUS doesn't work for kernels which include
MODULAR, but not atabus - such as macppc and evbarm
 1.135  08-Oct-2017  jdolecek as a stopgap measure, include necessary parts of dev/ata/ata.c even
if no atabus is configured, so that kernels with just 'wd* at umass?'
and no atabus compile; fixes kernel builds for vax, pmax and evbmips
after NCQ branch merge

XXX wd* at umass? still actually untested after the merge, need to discuss
XXX what to do with it
 1.134  08-Oct-2017  mlelstv Don't assert channel lock in polled mode.

ahcisata (and maybe others) use ata_delay during initialization where
lock isn't taken.
 1.133  07-Oct-2017  jdolecek Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.132  10-Sep-2014  matt branches: 1.132.8;
g/u_int[0-9]*_t/ s/u_int/uint/g
 1.131  25-Jul-2014  dholland Add d_discard to all struct cdevsw instances I could find.

All have been set to "nodiscard"; some should get a real implementation.
 1.130  16-Mar-2014  dholland branches: 1.130.2;
Change (mostly mechanically) every cdevsw/bdevsw I can find to use
designated initializers.

I have not built every extant kernel so I have probably broken at
least one build; however I've also found and fixed some wrong
cdevsw/bdevsw entries so even if so I think we come out ahead.
 1.129  12-Oct-2013  christos Pass the device name in, so we can debug what deferred drivers did not work.
 1.128  07-Aug-2013  blymn Make sure a channel has drives attached before resetting it otherwise
wdc_reset will panic the kernel due to a null derefrence of ch_drives.
 1.127  03-Apr-2013  bouyer branches: 1.127.4;
Fix kernel dump on ahci controller, by making sure we won't sleep
while dumping:
- introduce ata_delay() which either use delay() or kpause()
depending on flags. use it in sata_reset_interface() and
some ahci functions
- kill ATA_NOSLEEP, it was tested but never set. use ATA_POLL instead.
- reduce delay while polling in ahci, to speed up the dump

Should fix PR kern/41095
 1.126  01-Nov-2012  abs Allow setting ATADEBUG_MASK to default value of atadebug_mask
 1.125  04-Aug-2012  bouyer branches: 1.125.2;
If ch_ndrives is > 0, then ch_drive is not supposed to be NULL.
 1.124  31-Jul-2012  bouyer Apply back changes that were reverted on Jul 24 and Jul 26 (general ata/wdc
cleanup and SATA PMP support), now that I'm back to fix the fallouts.
 1.123  29-Jul-2012  jakllsch Remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas. Originally part of a commit by bouyer@.
 1.122  26-Jul-2012  jakllsch Revert, with intention of restoring in a less invasive way, the SATA Port
Multiplier code.

ok christos@
 1.121  24-Jul-2012  jakllsch Revert dsl@'s changes of Sun, 15 Jul 2012 10:55:35 +0000 and
Sun, 15 Jul 2012 10:56:50 +0000, excepting the kernel version bump.
First step in reverting regressions to ata(4) subsystem during the addition of
port multiplier support.
 1.120  22-Jul-2012  jakllsch It turns out my previous commit here was off by one.
 1.119  15-Jul-2012  jakllsch Stopgap crash prevention when atadebug_mask includes DEBUG_PROBE bit.
 1.118  15-Jul-2012  dsl Some namespace protection (and add greppablity).
Prefix the DRIVE_ and DRIVET_ constants from atavar.h with ATA_.
Don't use an enum for drive_type - you don't know how big it will be.
Move driver_type to avoid implicit structure padding (esp on arm).
This change is purely lexical and mechanical.

Update to 6.99.9 - this wasn't done when the SATA PMP changes
were made - I'm sure they warranted a bump.
 1.117  02-Jul-2012  bouyer Add sata Port MultiPlier (PMP) support to the ata bus layer,
as described in
http://mail-index.netbsd.org/tech-kern/2012/06/23/msg013442.html
PMP support in integrated to the atabus layer.
struct ata_channel's ch_drive[] is not dynamically allocated, and ch_ndrive
(renamed to ch_ndrives) closely reflects the size of the ch_drive[] array.
Add helper functions atabus_alloc_drives() and atabus_free_drives()
to manage ch_drive[]/ch_ndrives.
Add wdc_maxdrives to struct wdc_softc so that bus front-end can specify
how much drive they really support (master/slave or single).
ata_reset_drive() callback gains a uint32_t *sigp argument which,
when not NULL, will contain the signature of the device being reset.
While there, some cosmetic changes:
- added a drive_type enum to ata_drive_datas, and stop encoding the
probed drive type in drive_flags (we were out of drive flags anyway).
- rename DRIVE_ATAPIST to DRIVE_ATAPIDSCW to better reflect what this
really is
- remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas
- factor out the interpretation of SATA signatures in sata_interpet_sig()

propagate these changes to the ATA HBA drivers, and add support for PMP
to ahcisata(4) and siisata(4).

Thanks to:
- Protocase (http://www.protocase.com/) which provided a system
with lots of controllers, SATA PMP and drive slots
- Conservation Genomics Laboratory, Department of Biology, New Mexico State
University for hosting the above system
- Brook Milligan, who set up remote access and has been very responsive
when SATA cable move was needed
 1.116  06-Apr-2012  isaki Fix a typo (that I found 5 years ago). Though the definition of
atabusiodetach_args is the same as atabusioscan_args (at the moment).
 1.115  30-Apr-2011  jakllsch branches: 1.115.4; 1.115.8;
Add simplistic atabus(4) rescan support.
 1.114  18-Apr-2011  rmind Replace few simple_lock and ltsleep/wakeup uses with mutex(9) and condvar(9).

Note to all: please replace old primitives in your code! Thanks.
 1.113  28-Mar-2010  snj branches: 1.113.2;
Spell "enough" properly.
 1.112  24-Feb-2010  dyoung branches: 1.112.2;
A pointer typedef entails trading too much flexibility to declare const
and non-const types, and the kernel uses both const and non-const
PMF qualifiers and device suspensors, so change the pmf_qual_t and
device_suspensor_t typedefs from "pointers to const" to non-pointer,
non-const types.
 1.111  21-Jan-2010  dyoung branches: 1.111.2;
Cosmetic: join some lines. Get out early on errors, change

switch (...) {
...
error = ...;
break;
}
return error;

to

switch (...) {
...
return ...;
}
 1.110  08-Jan-2010  dyoung Expand PMF_FN_* macros.
 1.109  12-Nov-2009  dyoung Remove superfluous activation hook.
 1.108  19-Oct-2009  bouyer Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !
 1.107  13-Sep-2009  pooka Wipe out the last vestiges of POOL_INIT with one swift stroke. In
most cases, use a proper constructor. For proplib, give a local
equivalent of POOL_INIT for the kernel object implementation. This
way the code structure can be preserved, and a local link set is
not hazardous anyway (unless proplib is split to several modules,
but that'll be the day).

tested by booting a kernel in qemu and compile-testing i386/ALL
 1.106  12-May-2009  cegger struct cfdata * -> cfdata_t, no functional changes intended.
 1.105  07-Apr-2009  dyoung Cosmetic: flag a potential problem in ata_channel_attach():
it initializes a callout that does not appear to be destroyed
anywhere. Every callout_init() should ordinarily be matched by a
callout_destroy().
 1.104  03-Apr-2009  dyoung Stop dereferencing a dangling device_t pointer and crashing: skip the
drives flagged DRIVE_ATAPI in atabus_activate(,DVACT_DEACTIVATE) just as
we skip them in atabus_detach() and in atabus_childdetched().

Make atabus_detach() parallel attachment more closely by calling
config_detach() on the child chp->ata_drives[i] instead of on
chp->ch_drive[i].drv_softc. Assert that ata_drives[i] and
ch_drive[i].drv_softc are equal, and set them both to NULL in
atabus_childdetached().
 1.103  02-Apr-2009  dyoung During shutdown, detach devices in an orderly fashion.

Call the detach routine for every device in the device tree, starting
with the leaves and moving toward the root, expecting that each
(pseudo-)device driver will use the opportunity to gracefully commit
outstandings transactions to the underlying (pseudo-)device and to
relinquish control of the hardware to the system BIOS.

Detaching devices is not suitable for every shutdown: in an emergency,
or if the system state is inconsistent, we should resort to a fast,
simple shutdown that uses only the pmf(9) shutdown hooks and the
(deprecated) shutdownhooks. For now, if the flag RB_NOSYNC is set in
boothowto, opt for the fast, simple shutdown.

Add a device flag, DVF_DETACH_SHUTDOWN, that indicates by its presence
that it is safe to detach a device during shutdown. Introduce macros
CFATTACH_DECL3() and CFATTACH_DECL3_NEW() for creating autoconf
attachments with default device flags. Add DVF_DETACH_SHUTDOWN
to configuration attachments for atabus(4), atw(4) at cardbus(4),
cardbus(4), cardslot(4), com(4) at isa(4), elanpar(4), elanpex(4),
elansc(4), gpio(4), npx(4) at isa(4), nsphyter(4), pci(4), pcib(4),
pcmcia(4), ppb(4), sip(4), wd(4), and wdc(4) at isa(4).

Add a device-detachment "reason" flag, DETACH_SHUTDOWN, that tells the
autoconf code and a device driver that the reason for detachment is
system shutdown.

Add a sysctl, kern.detachall, that tells the system to try to detach
every device at shutdown, regardless of any device's DVF_DETACH_SHUTDOWN
flag. The default for kern.detachall is 0. SET IT TO 1, PLEASE, TO
HELP TEST AND DEBUG DEVICE DETACHMENT AT SHUTDOWN.

This is a work in progress. In future work, I aim to treat
pseudo-devices more thoroughly, and to gracefully tear down a stack of
(pseudo-)disk drivers and filesystems, including cgd(4), vnd(4), and
raid(4) instances at shutdown.

Also commit some changes that are not easily untangled from the rest:

(1) begin to simplify device_t locking: rename struct pmf_private to
device_lock, and incorporate device_lock into struct device.

(2) #include <sys/device.h> in sys/pmf.h in order to get some
definitions that it needs. Stop unnecessarily #including <sys/device.h>
in sys/arch/x86/include/pic.h to keep the amd64, xen, and i386 releases
building.
 1.102  16-Nov-2008  bouyer branches: 1.102.4;
cpu_intr_p() doesn't account for software interrupts (e.g. callouts) so
we can't use it here. Rssurect ATACH_TH_RUN, backing out
src/sys/dev/ata/ata.c 1.101
src/sys/dev/ata/ata_wdc.c 1.90
src/sys/dev/ata/atavar.h 1.77
src/sys/dev/ic/wdc.c 1.255
src/sys/dev/scsipi/atapi_wdc.c 1.108
Should fix kern/39927 and kern/39725.
 1.101  02-Oct-2008  bouyer branches: 1.101.2; 1.101.4;
Kill ATACH_TH_RUN and use cpu_intr_p() instead.
 1.100  01-Oct-2008  bouyer Use a separate thread to probe/attach atabus's childrens. Fixes a deadlock
where the interrupt routine wants to wake up the atabus thread to perform a
reset, while the thread is blocked in wd's attach function.
 1.99  12-Jun-2008  cegger branches: 1.99.2;
use device_lookup_private to get softc
 1.98  18-Mar-2008  cube branches: 1.98.2; 1.98.4; 1.98.6; 1.98.8;
Split device_t and softc for ATA devices, as well as wd(4). Other
cosmetic changes where appropriate.
 1.97  29-Feb-2008  dyoung Use PMF_FN_ARGS, PMF_FN_PROTO.
 1.96  10-Jan-2008  dyoung branches: 1.96.2; 1.96.6;
Oops, remove some debug statements that slipped in there.
 1.95  10-Jan-2008  dyoung Use device_t, device_xname(), device_private(). Add atabus_childdetached()
to clean up pointers to detached children.
 1.94  04-Jan-2008  ad Start detangling lock.h from intr.h. This is likely to cause short term
breakage, but the mess of dependencies has been regularly breaking the
build recently anyhow.
 1.93  20-Dec-2007  pooka Don't allocate DEV_BSIZE from the stack, use kmem instead.
 1.92  09-Dec-2007  jmcneill branches: 1.92.2;
Merge jmcneill-pm branch.
 1.91  19-Oct-2007  ad branches: 1.91.2; 1.91.4; 1.91.6;
machine/{bus,cpu,intr}.h -> sys/{bus,cpu,intr}.h
 1.90  09-Jul-2007  ad branches: 1.90.6; 1.90.8; 1.90.12;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements
 1.89  01-Jul-2007  dsl Remove 'else' after 'return' for clarity
 1.88  17-Mar-2007  dyoung Let config(1) know that #define ATADEBUG goes in opt_ata.h. In
dev/ic/wdc.c and in dev/ata/ata.c, #include "opt_ata.h", and make
both the files compile with or *without* ATADEBUG. Do not compile
with ATADEBUG by default.
 1.87  12-Mar-2007  ad branches: 1.87.2; 1.87.4;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.
 1.86  04-Mar-2007  christos branches: 1.86.2;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
 1.85  09-Feb-2007  ad branches: 1.85.2;
Merge newlock2 to head.
 1.84  25-Dec-2006  wiz Spell "schedule" correctly. From Zafer Aydogan.
 1.83  16-Nov-2006  christos __unused removal on arguments; approved by core.
 1.82  25-Oct-2006  bouyer Add sata registers to struct wdc_regs. Add wdc_sataprobe(), a function
probing drives using the standard SATA registers; taken from various
PCI sata drivers. Export wdc_drvprobe() too.
 1.81  15-Oct-2006  itohy white space police
 1.80  14-Oct-2006  itohy Add ATAPI Pioneer (info from Linux) and ATA TDK (I have one)
as exceptions of string byte order.
Add string byte order shuffling for big-endian platforms.
 1.79  12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.78  30-Sep-2006  itohy A little effort against kernel bloat....
Exclude ATA DMA support if no ATA DMA capable drivers are compiled in.
 1.77  24-Sep-2006  jmcneill Add "name" parameter to powerhook_establish, to aid debugging. No objections
on tech-kern@
 1.76  07-Sep-2006  itohy branches: 1.76.2; 1.76.4;
Add PIOBM (busmastering transfer using ATA PIO mode) support.
The PIOBM is used by only one driver (will be added later,
stay tuned) and intruduce an attribute "ata_piobm" so that
it will be conditionally compiled in.
The "ata_dma" (busmastering transfer using ATA DMA mode) and
"ata_udma" (busmastering transfer using ATA Ultra DMA mode)
attributes are also added for consistency, but unused for now.
 1.75  27-Aug-2006  christos Add missing initializer
 1.74  29-Mar-2006  thorpej Use device_cfdata().
 1.73  22-Jan-2006  bouyer branches: 1.73.2; 1.73.4; 1.73.6; 1.73.8; 1.73.10;
For an IDENTIFY we have to wait for DRQ after issuing the command.
 1.72  11-Dec-2005  christos branches: 1.72.2;
merge ktrace-lwp.
 1.71  07-Jul-2005  drochner tighten the autoconf constraints by passing the atapi/ata/ata_hl attribute
to config_found()
(after some cleanup we might be able to kill SCSIPI_BUSTYPE_*)
 1.70  29-May-2005  christos branches: 1.70.2;
- remove casts [to avoid volatile]
- sprinkle const
- avoid shadowed variables
 1.69  20-May-2005  bouyer Close a window where (disks) interrupts could be enabled while the
ATACH_TH_RUN flag is still set. Should fix panic reported by Jukka Salmi
on netbsd-users@.
 1.68  16-May-2005  bouyer Add a ata_queue_idle() function, which freeze a queue and tsleep() until the
controller is idle.
Change the powerhook function to call ata_queue_idle() on standby/suspend,
and ata_reset_channel() on resume (to wake up the disks and start from a
clean state).
Fix PR kern/30194 by Lubomir Sedlacik.
 1.67  11-Apr-2005  matt Fix mode printing. (don't panic on unmatched devices)
 1.66  04-Mar-2005  tacha branches: 1.66.2;
Disestablish powerhook while detaching atabus.
 1.65  27-Feb-2005  perry nuke trailing whitespace
 1.64  26-Jan-2005  jmcneill Add powerhook for atabus.
 1.63  14-Nov-2004  soren branches: 1.63.4; 1.63.6;
s/using DMA data transfers/using DMA/ to make it fit in 80 columns.
DMA only applies to data transfers anyway..
 1.62  30-Oct-2004  bouyer Allow polled commands to be queued, if ata_exec_xfer() is called in thread
context. For this introduce 3 new xfer tags:
- C_WAIT, equivalent of AT_WAIT
- C_WAITACT, when there is a thread waiting on this xfer to become the active
one (that is, to be at the head of the queue)
- C_FREE, set by ata_free_xfer() when it can't free the xfer because it's
still in use. The holder should then free the xfer ASAP.

If ata_exec_xfer() is called with (C_POLL | C_WAIT), and there is already
xfers to be processed, assert C_WAITACT and sleep.
atastart() checks for C_WAITACT, and wakeup the thread waiting for this xfer
to become active if set. atastart() won't process this xfer, it's the
responsability of the thread waked up to handle it.

Fix (the right way) kern/27421 by Martin Husemann.
 1.61  28-Oct-2004  bouyer Call xfer->c_start() with the channel from the xfer, not the one from the
thread. They may be different if we have a shared queue.
 1.60  11-Sep-2004  bouyer If we're not going to handle a reset request because one if already pending,
decrease queue_freeze that we just increased (otherwise the queue stay
frozen after the reset).
Should fix kern/26873 and port-i386/26910.
 1.59  21-Aug-2004  thorpej Eliminate some gratuitous spl frobbing in atabus_thread().
 1.58  21-Aug-2004  thorpej atastart() (called only at splbio(), and from interrupts) can change
drive_flags, to make sure all drive_flags manipulations are done at
splbio().
 1.57  20-Aug-2004  thorpej - atabus_detach(): Manipulate ch_flags at splbio().
- ata_reset_channel(): Assert that we're at splbio() when called.
- Sprinkle some comments about functions that must be called at splbio().
 1.56  20-Aug-2004  thorpej WDC_DIAGNOSTIC -> ATA_DEBUG
 1.55  20-Aug-2004  thorpej - Add an (*ata_reset_channel)() member to ata_bustype.
- Add an ata_reset_channel() function that performs the common parts
of resetting an ATA channel, which uses the (*ata_reset_channel)()
callback to do the heavy lifting. Adjust callers to use ata_reset_channel()
instead of wdc_reset_channel().

This removes the last wdc-specific code from ata.c!
 1.54  20-Aug-2004  thorpej Use ANSI function decls for atabus open/close/ioctl.
 1.53  20-Aug-2004  thorpej wdcreg.h is not needed here any more.
 1.52  20-Aug-2004  thorpej Move common parts of attaching an ata_channel to a new ata_channel_attach()
function.
 1.51  20-Aug-2004  thorpej Move atabusconfig() to ata.c.
 1.50  20-Aug-2004  thorpej In ata_get_params(), use the (*ata_exec_command)() available from the
ata_bustype rather than using wdc_exec_command() directly.
 1.49  20-Aug-2004  thorpej Move most of wdc_softc into a new atac_softc structure that contains
info common to all types of ATA controllers.
 1.48  14-Aug-2004  thorpej - Split the register handles out of struct wdc_channel into a separate
wdc_regs structure, and array of which (indexed per channel) is pointed
to by struct wdc_softc.
- Move the resulting wdc_channel structure to atavar.h and rename it to
ata_channel. Rename the corresponding flags.
- Add a "ch_ndrive" member to struct ata_channel, which indicates the
maximum number of drives that can be present on the channel. For now,
this is always 2. Add an ATA_MAXDRIVES constant that places an upper
limit on this value, also currently 2.
 1.47  13-Aug-2004  thorpej WDCDEBUG -> ATADEBUG.
 1.46  13-Aug-2004  thorpej Don't bother with bits that tell of the presence of optional callbacks;
just check the function pointers for NULL.
 1.45  13-Aug-2004  thorpej Move wdcstart() to ata.c and rename it to atastart().
 1.44  13-Aug-2004  thorpej Move wdc_exec_xfer() to ata.c and rename it ata_exec_xfer().
 1.43  12-Aug-2004  thorpej Move wdc_addref() and wdc_delref() to ata.c and rename them to
ata_*.
 1.42  12-Aug-2004  thorpej Move wdc_kill_pending() to ata.c and rename it ata_kill_pending().
 1.41  12-Aug-2004  thorpej - Move wdc_xfer_pool, wdc_get_xfer(), wdc_free_xfer() to ata.c, and rename
to ata_*.
- Use a static initializer for the ata_xfer_pool.
 1.40  12-Aug-2004  thorpej Move wdc_probe_caps() to ata.c and rename it ata_probe_caps().
 1.39  12-Aug-2004  thorpej Move wdc_downgrade_mode() to ata.c and rename it to ata_downgrade_mode().
 1.38  12-Aug-2004  thorpej Move wdc_print_modes() into ata.c and rename it ata_print_modes().
 1.37  12-Aug-2004  thorpej Rename some constants:
WDC_COMPLETE -> ATACMD_COMPLETE
WDC_QUEUED -> ATACMD_QUEUED
WDC_TRY_AGAIN -> ATACMD_TRY_AGAIN
 1.36  12-Aug-2004  thorpej Rename "struct wdc_command" to "struct ata_command".
 1.35  10-Aug-2004  mycroft Hold a reference between atabus_attach() and atabusconfig() to avoid an extra
enable/disable cycle. Now my CF card attaches a bit faster.

XXX This code is split strangely between files.
 1.34  04-Aug-2004  bouyer Change wdc_kill_pending() to take a struct ata_drive_datas * as argument,
and kill only pending requests for this drive.
Implement a DRIVE_WAITDRAIN flag, which will cause the active command to
be killed once complete.
Other minor fixes.
Now it's possible to detach a ATA or ATAPI device from ioctl even when
a dd on the raw char partition is running.
 1.33  04-Aug-2004  bouyer Add
struct ata_xfer *active_xfer
to ata_queue. Now the active xfer isn't the head of the queue any more,
this makes a few things easier (this will also help for tagged queuing
support).
Remove the WDCF_ACTIVE flag, test active_xfer != NULL instead.
clean up wdc_free_xfer() and kill_xfer().
Clean up wdc_reset_channel(), and make it issue a ATAPI_SOFT_RESET if the
active command is ATAPI.
In wdc_atapi_get_params(), use AT_WAIT | AT_POLL for ATAPI_SOFT_RESET,
so that we'll use tsleep() instead of delay().
In wdc_atapi_start(), call wdc_dmawait() at the right place.
 1.32  03-Aug-2004  bouyer First cut at implementing ATABUSIODETACH. Still buggy.
 1.31  02-Aug-2004  bouyer Clean up interraction between wdc_reset_channel and the kernel thread.
Move kill_xfer() after the reset, and stop the DMA engine if needed
(this will unload the DMA maps).
 1.30  01-Aug-2004  bouyer Implement an atabus control device, and define some ATA bus control
IOCTLS. Implement ATABUSIORESET, which will reset the given ATA bus.
 1.29  27-May-2004  thorpej Clean-up of use of "precomp". It is used as "features" everywhere, so
rename the fields in wdc_command as appropriate.
 1.28  13-Apr-2004  bouyer Make wdc_channel->ch_flags volatile, and cast it to (void *) when used
in tsleep()/wakeup() to appease gcc. Otherwise, the ch_flags value may
be cached in a register in atabus_thread(), and when it sets the WDCF_TH_RUN
bit after tsleep() it may loose loose the changes made by an interrupt handler
or another thread.
Problem analysed by Jukka Andberg on tech-kern.
 1.27  03-Jan-2004  thorpej branches: 1.27.2;
More structure member namespace cleanup: thread -> ch_thread
 1.26  03-Jan-2004  thorpej More wdc_channel structure member namespace cleanup:
- channel -> ch_channel
- wdc -> ch_wdc
 1.25  03-Jan-2004  thorpej Rename "struct channel_softc" to "struct wdc_channel".
 1.24  01-Jan-2004  thorpej Rename:
- wdc_xfer to ata_xfer
- channel_queue to ata_queue
and move them to <dev/ata/atavar.h> so they can be used by non-wdc ATA
controllers. Clean up the member names of these structures while at it.
 1.23  30-Dec-2003  thorpej Move most of the atabus layer into ata.c.
 1.22  30-Dec-2003  thorpej Tweak function names in debug messages.
 1.21  14-Dec-2003  thorpej De-__P and use ANSI-style function decls.
 1.20  08-Oct-2003  bouyer Make the ATA mid-layer appears as atabus, as proposed in
http://mail-index.netbsd.org/tech-kern/2003/09/25/0006.html
This adds a device (atabus) between IDE controllers and wd or atapibus, to
have each ATA channel show up in the device tree. Later there will be atabus
devices in /dev, so that we can do IOCTL on them.
Each atabus has its own kernel thread, to handle operations that needs polling,
e.g. reset and others.

Device probing on each bus it defered to the atabus thread creation.
This allows to do the reset and basic device probes in parallel, which reduce
boot time on systems with several pciide controllers.
 1.19  23-Sep-2003  mycroft Fix more probe delay and/or failure problems:
1) Don't wait for DRQ on an IDENTIFY command -- if it's not set when we see
BSY clear, abort the command and ignore the drive. (Do this by testing
for DRQ in the read/write cases in __wdccommand_intr().)
2) Don't wait for DRQ to deassert when we finish an IDENTIFY (or any other
non-block command that reads data) -- we don't do this for block I/O, and
empirically it doesn't clear on my CF cards at all, causing a pointless 1s
delay.
3) Add comments to some of the delay()s, and add missing ones in wdcreset()
and the WDCC_RECAL in the so-called "pre-ATA" probe.
4) Slightly simplify the reset sequence -- we were doing an extra I/O.
5) Modify the register writability test to make sure that registers are not
overlapped -- this can happen in some weird cases with a missing device 1.
6) Check the error register value after the reset -- if it's not 01h or 81h,
as appropriate (see ATA spec), punt.
Tested with a number of ATA-only, ATAPI-only, mixed ATA-ATAPI, CF, and IDE
disk configurations.

Also remove the SINGLE_DRIVE nonsense again.
 1.18  27-Jan-2003  bouyer branches: 1.18.2;
Bump WDCC_IDENTIFY timeout to 3 seconds, as required by some drives.
Reported by Karl Janmar <karlj@mdstud.chalmers.se> on tech-kern.
 1.17  09-Apr-2002  bouyer branches: 1.17.4;
In wdc_exec_command(), for data commands, read the status register after
the data transfer. This is mandatory for data out commands (although none are
used for now), and not forbiddend for data in commands. Also record if we
did transfers any data.
May solve kern/16159 by making the probe more robust in face of fake identify.
 1.16  03-Dec-2001  bouyer Update my copyrigth.
 1.15  03-Dec-2001  bouyer Allow wd to attach to something else than wdc/pciide (like USB<->ATA bridges):
- move some functions from ata.c to ata_wdc.c or wdc.c.
- add callbacks to struct ata_bustype so that wd.c doesn't call directly
functions from the lower level driver.
 1.14  13-Nov-2001  lukem add RCSID
 1.13  15-May-2000  bouyer branches: 1.13.4; 1.13.6; 1.13.8;
- sync my copyrigth
- Add usefull message for obsolete error code (part of fix for kern/9856).
 1.12  25-Jan-2000  enami Don't put leading space, otherwise diskerr() prints unnecessary space
like this:
wd0h: aborted command reading fsbn 650256 of 650256-650271 (wd0 bn 839056; cn 888 tn 13 sn 17), retrying
 1.11  17-Jan-2000  bouyer Improve the downgrade logic:
- If UDMA 2 is failing try UDMA 1 first, it helps in some cases
- downgrade if we get an error in the first 4000 xfers, or if we get
4 errors in 4000 xfers if the first 4000 went without troubles.

While I'm there commit a local change I have since some time to get my CD
probed: issue a "blanck" IDENTIFY before the one used to detect slave ghosts,
with my drive the first IDENTIFY following a controller reset fails with an
aborted command ...
 1.10  26-Nov-1999  bouyer More WDCDEBUG_PRINT.
 1.9  15-Apr-1999  bouyer branches: 1.9.2; 1.9.8;
Bump timeout to 10s only for ATAPI devices, 1s is fine for IDE drives once
they are ready.
 1.8  14-Apr-1999  jonathan Complete bugfix in rev 1.3: increase read-parameter-block timeout to 10s
(as used for the hardware for which rev 1.3 was made.)
 1.7  10-Mar-1999  bouyer branches: 1.7.2;
Add a new disk flag: DRIVE_OLD for pre-ATA disks. probe routine will now set
DRIVE_OLD, DRIVE_ATA or DRIVE_ATAPI based on register signatures.
The attach routine will issue a IDENTIFY command for ATA/ATAPI disk,
to detect flase matches by the probe routine.
probe/attach should now be fully compliant with ata-4/ata-5. As a side
effect, ATAPI drives which improperly use ATA register signatures should now
be attached as ATAPI.
 1.6  21-Feb-1999  hubertf Only define WDCDEBUG if not already define (e.g. by kernel option)
 1.5  08-Feb-1999  bouyer There's no ATA draft where it is required for the drive to set DRDY | DSC when
the disk is ready to transfer data, and in ATA-5 the DSC has been obsoleted.
So only wait for DRQ to transfer data. This can be made conditional on the
ATA version if it's proven to break with some drives (worked with all the
drives I have access to).
While I'm there correct a few typos.
 1.4  18-Jan-1999  bouyer move wd.c:print_wderror() to ata.c:ata_perror().
In wdc_probe_caps() add code to guess the ATA revision supported (if
ATA4 if Ultra-DMA, ATA2 if PIO mode > 2). We can't rely on param.atap_ata_major
here, at last one Ultra-DMA drive claims to support only ATA-3.
Use the ATA version in ata_perror(), and to try a flush cache command
in a shutdown hook for IDE drives.
 1.3  16-Nov-1998  bouyer For ATAPI IDENTiFY, poll only for DRQ, some devices don't assert DRDY at this
point.
 1.2  12-Oct-1998  bouyer Merge bouyer-ide
 1.1  04-Jun-1998  bouyer branches: 1.1.2;
file ata.c was initially added on branch bouyer-ide.
 1.1.2.4  20-Sep-1998  bouyer - always call config_found() from wdcattach(), instead of printing our own
the "not configured" message ourselve. When no atapibus is configured,
use a fake ata_atapi_attach, else call wdc_atapibus_attach(). This way we
don't have to include the whole atapi support in an ata-only config.
- the dma_* function pointers take a 'int flags' as last argument, instead
of an 'int read'. The 2 valid flags are: WDC_DMA_READ (to select read/write)
and WDC_DMA_POLL, to signal interrupt-less mode (for core dumps).
- Reworked wdcprobe() so look more like the old one. A status of 0xff is
interpreted as "no drive" (freebsd does this); this this speed up
the probe for non-IDE machines (a reset timeout has been bumped to 31s,
to match the specs). The probe set ups the drive flags to either ATA or
ATAPI (depend on register signature) or OLD (if register signature don't
match, but a working drive appears to be there). Later the ATA code
will look at the OLD flag to decide wether there is no drive, or an old
(ST506) one if WDC_IDENTIFY fails.
- For known PCI chips, don't use the wdcprobe() heuristic to find wether
a channel is enabled/disabled; use the chip's specific registers for this.
- Clear the command queue when we have a polled command. This allows dumps to
work even if a command was being handled by the controller at panic time.
- Increase the number of available DMA segments by one, as user requests may
not be aligned on a page boundary.
 1.1.2.3  13-Aug-1998  bouyer - sync with HEAD
- better error detection and report for wd drives
- better wddump() support
- wdcwait() now takes a timout argument, so that we can honnor the
timeout argument of scsipi cmds.
 1.1.2.2  05-Jun-1998  bouyer Patches from Leo Weppelman for IDE on atari, with minor fixes by me
(needed the addition of a void wdcrestart __P((void*)) function).
Quoting his words:
These diffs are preliminary because it looks like the free_hw() function
does not get called. This is only a problem on the atari falcon. I have not
yet been able to track this down.
Also, KNF wdc.c.
 1.1.2.1  04-Jun-1998  bouyer Commit changes to the IDE system in a branch. This allows a better separation
between higth-level and low-level (i.e. registers read/write) and generalize
the queue for all commands. This also add supports for IDE DMA.
 1.7.2.3  07-Jul-2000  he Apply patch (requested by bouyer):
Add support for the following PCIIDE controllers:
o AMD 756
o CMD PCI0648 and PCI0649
o Hightpoint HPT366
o OPTi 82c621 (and a few of its derivatives)
o Promise Ultra/33 and Ultra/66
o Intel 82801 (ICH/ICH0)
Also fix PR#10437 (detect more ATAPI devices).
 1.7.2.2  23-Jan-2000  he Pull up revision 1.11 (requested by bouyer):
Improve the downgrade logic to work better in some cases (especially
for Acer Labs M5229 controllers with Ultra-DMA/66 disks).
 1.7.2.1  20-Apr-1999  cjs branches: 1.7.2.1.2;
Pull up to HEAD 1.9, per Manuel Bouyer <bouyer@antioche.eu.org>
 1.7.2.1.2.1  21-Jun-1999  thorpej Sync w/ -current.
 1.9.8.2  27-Dec-1999  wrstuden Pull up to last week's -current.
 1.9.8.1  21-Dec-1999  wrstuden Initial commit of recent changes to make DEV_BSIZE go away.

Runs on i386, needs work on other arch's. Main kernel routines should be
fine, but a number of the stand programs need help.

cd, fd, ccd, wd, and sd have been updated. sd has been tested with non-512
byte block devices. vnd, raidframe, and lfs need work.

Non 2**n block support is automatic for LKM's and conditional for kernels
on "options NON_PO2_BLOCKS".
 1.9.2.1  20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
A i386 GENERIC kernel compiles without the siop, ahc and bha drivers
(will be updated later). i386 IDE/ATAPI and ncr work, as well as
sparc/esp_sbus. alpha should work as well (untested yet).
siop, ahc and bha will be updated once I've updated the branch to current
-current, as well as machine-dependant code.
 1.13.8.2  23-Jun-2002  jdolecek catch up with -current on kqueue branch
 1.13.8.1  10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.13.6.3  17-Apr-2002  nathanw Catch up to -current.
 1.13.6.2  08-Jan-2002  nathanw Catch up to -current.
 1.13.6.1  14-Nov-2001  nathanw Catch up to -current.
 1.13.4.1  23-Jan-2003  msaitoh Pull up revision 1.17 (requested by Manuel Bouyer):

read status after data commands in wdc_exec_command(). Mandatory for data out
commands (nonw are used for now), and doesn't hurt for data in. Also keep
track if we did transfer data. Makes the probe more robust and solves
kern/16159
 1.17.4.1  30-Jun-2003  grant Pull up revision 1.18 (requested by bouyer in ticket #1353):

Bump WDCC_IDENTIFY timeout to 3 seconds, as required by some drives.
Reported by Karl Janmar <karlj@mdstud.chalmers.se> on tech-kern.
 1.18.2.10  10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.18.2.9  04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.18.2.8  04-Feb-2005  skrll Sync with HEAD.
 1.18.2.7  29-Nov-2004  skrll Sync with HEAD.
 1.18.2.6  02-Nov-2004  skrll Sync with HEAD.
 1.18.2.5  21-Sep-2004  skrll Fix the sync with head I botched.
 1.18.2.4  18-Sep-2004  skrll Sync with HEAD.
 1.18.2.3  25-Aug-2004  skrll Sync with HEAD.
 1.18.2.2  12-Aug-2004  skrll Sync with HEAD.
 1.18.2.1  03-Aug-2004  skrll Sync with HEAD
 1.27.2.2  24-May-2005  riz Pull up revision 1.69 via patch(requested by bouyer in ticket #1543):
Close a window where (disks) interrupts could be enabled while the
ATACH_TH_RUN flag is still set. Should fix panic reported by Jukka Salmi
on netbsd-users@.
 1.27.2.1  18-Apr-2004  jmc branches: 1.27.2.1.2;
Pullup rev 1.28 (requested by bouyer in ticket #149)

Make wdc_channel->ch_flags volatile, and cast it to (void *) when used
in tsleep()/wakeup() to appease gcc. Otherwise, the ch_flags value may
be cached in a register in atabus_thread(), and when it sets the WDCF_TH_RUN
bit after tsleep() it may loose loose the changes made by an interrupt handler
or another thread.
 1.27.2.1.2.1  24-May-2005  riz Pull up revision 1.69 via patch(requested by bouyer in ticket #1543):
Close a window where (disks) interrupts could be enabled while the
ATACH_TH_RUN flag is still set. Should fix panic reported by Jukka Salmi
on netbsd-users@.
 1.63.6.2  19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.63.6.1  12-Feb-2005  yamt sync with head.
 1.63.4.1  29-Apr-2005  kent sync with -current
 1.66.2.2  28-May-2005  tron Pull up revision 1.69 (requested by bouyer in ticket #340):
Close a window where (disks) interrupts could be enabled while the
ATACH_TH_RUN flag is still set. Should fix panic reported by Jukka Salmi
on netbsd-users@.
 1.66.2.1  27-May-2005  riz Pull up revision 1.68 (requested by bouyer in ticket #328):
Add a ata_queue_idle() function, which freeze a queue and tsleep() until the
controller is idle.
Change the powerhook function to call ata_queue_idle() on standby/suspend,
and ata_reset_channel() on resume (to wake up the disks and start from a
clean state).
Fix PR kern/30194 by Lubomir Sedlacik.
 1.70.2.8  24-Mar-2008  yamt sync with head.
 1.70.2.7  17-Mar-2008  yamt sync with head.
 1.70.2.6  21-Jan-2008  yamt sync with head
 1.70.2.5  27-Oct-2007  yamt sync with head.
 1.70.2.4  03-Sep-2007  yamt sync with head.
 1.70.2.3  26-Feb-2007  yamt sync with head.
 1.70.2.2  30-Dec-2006  yamt sync with head.
 1.70.2.1  21-Jun-2006  yamt sync with head.
 1.72.2.1  01-Feb-2006  yamt sync with head.
 1.73.10.1  31-Mar-2006  tron Merge 2006-03-31 NetBSD-current into the "peter-altq" branch.
 1.73.8.1  19-Apr-2006  elad sync with head.
 1.73.6.3  14-Sep-2006  yamt sync with head.
 1.73.6.2  03-Sep-2006  yamt sync with head.
 1.73.6.1  01-Apr-2006  yamt sync with head.
 1.73.4.1  22-Apr-2006  simonb Sync with head.
 1.73.2.1  09-Sep-2006  rpaulo sync with head
 1.76.4.2  10-Dec-2006  yamt sync with head.
 1.76.4.1  22-Oct-2006  yamt sync with head
 1.76.2.3  06-Feb-2007  ad Quieten noisy boot messages.
 1.76.2.2  12-Jan-2007  ad Sync with head.
 1.76.2.1  18-Nov-2006  ad Sync with head.
 1.85.2.2  24-Mar-2007  yamt sync with head.
 1.85.2.1  12-Mar-2007  rmind Sync with HEAD.
 1.86.2.8  23-Oct-2007  ad Sync with head.
 1.86.2.7  15-Jul-2007  ad Sync with head.
 1.86.2.6  01-Jul-2007  ad Adapt to callout API change.
 1.86.2.5  13-May-2007  ad - Pass the error number and residual count to biodone(), and let it handle
setting error indicators. Prepare to eliminate B_ERROR.
- Add a flag argument to brelse() to be set into the buf's flags, instead
of doing it directly. Typically used to set B_INVAL.
- Add a "struct cpu_info *" argument to kthread_create(), to be used to
create bound threads. Change "bool mpsafe" to "int flags".
- Allow exit of LWPs in the IDL state when (l != curlwp).
- More locking fixes & conversion to the new API.
 1.86.2.4  10-Apr-2007  ad Sync with head.
 1.86.2.3  10-Apr-2007  ad Nuke the deferred kthread creation stuff, as it's no longer needed.
Pointed out by thorpej@.
 1.86.2.2  09-Apr-2007  ad - Add two new arguments to kthread_create1: pri_t pri, bool mpsafe.
- Fork kthreads off proc0 as new LWPs, not new processes.
 1.86.2.1  13-Mar-2007  ad Sync with head.
 1.87.4.1  18-Mar-2007  reinoud First attempt to bring branch in sync with HEAD
 1.87.2.1  11-Jul-2007  mjf Sync with head.
 1.90.12.1  25-Oct-2007  bouyer Sync with HEAD.
 1.90.8.3  23-Mar-2008  matt sync with HEAD
 1.90.8.2  09-Jan-2008  matt sync with HEAD
 1.90.8.1  06-Nov-2007  matt sync with HEAD
 1.90.6.4  08-Dec-2007  jmcneill Rename pnp(9) -> pmf(9), as requested by many.
 1.90.6.3  06-Nov-2007  joerg Refactor PNP API:
- Make suspend/resume directly a device functionality. It consists of
three layers (class logic, device logic, bus logic), all of them being
optional. This replaces D0/D3 transitions.
- device_is_active returns true if the device was not disabled and was
not suspended (even partially), device_is_enabled returns true if the
device was enabled.
- Change pnp_global_transition into pnp_system_suspend and
pnp_system_resume. Before running any suspend/resume handlers, check
that all currently attached devices support power management and bail
out otherwise. The latter is not done for the shutdown/panic case.
- Make the former bus-specific generic network handlers a class handler.
- Make PNP message like volume up/down/toogle PNP events. Each device
can register what events they are interested in and whether the handler
should be global or not.
- Introduce device_active API for devices to mark themselve in use from
either the system or the device. Use this to implement the idle handling
for audio and input devices. This is intended to replace most ad-hoc
watchdogs as well.
- Fix somes situations in which audio resume would lose mixer settings.
- Make USB host controllers better deal with suspend in the light of
shared interrupts.
- Flush filesystem cache on suspend.
- Flush disk caches on suspend. Put ATA disks into standby on suspend as
well.
- Adopt drivers to use the new PNP API.
- Fix a critical bug in the generic cardbus layer that made D0->D3
break.
- Fix ral(4) to set if_stop.
- Convert cbb(4) to the new PNP API.
- Apply the PCI Express SCI fix on resume again.
 1.90.6.2  26-Oct-2007  joerg Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.
 1.90.6.1  03-Aug-2007  jmcneill Pull in power management changes from private branch.
 1.91.6.1  11-Dec-2007  yamt sync with head.
 1.91.4.1  26-Dec-2007  ad Sync with head.
 1.91.2.1  18-Feb-2008  mjf Sync with HEAD.
 1.92.2.3  10-Jan-2008  bouyer Sync with HEAD
 1.92.2.2  08-Jan-2008  bouyer Sync with HEAD
 1.92.2.1  02-Jan-2008  bouyer Sync with HEAD
 1.96.6.6  17-Jan-2009  mjf Sync with HEAD.
 1.96.6.5  05-Oct-2008  mjf Sync with HEAD.
 1.96.6.4  29-Jun-2008  mjf Sync with HEAD.
 1.96.6.3  06-Apr-2008  mjf - after some discussion with agc@ i agreed it would be a good idea to move
device_unregister_* to device_deregister_* to be more like the pmf(9)
functions, especially since a lot of the time the function calls are next
to each other.

- add device_register_name() support for dk(4).
 1.96.6.2  05-Apr-2008  mjf - add "file-system DEVFS" and "pseudo-device devfsctl" to conf/std seeing
as these are always needed.

- convert many, many drivers over to the New Devfs World Order. For a
list of device drivers yet to be converted see,
http://www.netbsd.org/~mjf/devfs-todo.html.

- add a new device_unregister_all(device_t) function to remove all device
names associated with a device_t, which saves us having to construct
device names when the driver is detached.

- add a DEV_AUDIO type for devices.
 1.96.6.1  03-Apr-2008  mjf Sync with HEAD.
 1.96.2.1  24-Mar-2008  keiichi sync with head.
 1.98.8.1  18-Jun-2008  simonb Sync with head.
 1.98.6.2  10-Oct-2008  skrll Sync with HEAD.
 1.98.6.1  23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.98.4.5  11-Aug-2010  yamt sync with head.
 1.98.4.4  11-Mar-2010  yamt sync with head
 1.98.4.3  16-Sep-2009  yamt sync with head
 1.98.4.2  16-May-2009  yamt sync with head
 1.98.4.1  04-May-2009  yamt sync with head.
 1.98.2.1  17-Jun-2008  yamt sync with head.
 1.99.2.2  13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.99.2.1  19-Oct-2008  haad Sync with HEAD.
 1.101.4.1  20-Nov-2008  snj Pull up following revision(s) (requested by bouyer in ticket #67):
sys/dev/ic/wdc.c: revision 1.256
sys/dev/ata/ata.c: revision 1.102
sys/dev/scsipi/atapi_wdc.c: revision 1.109
sys/dev/ata/atavar.h: revision 1.78
sys/dev/ata/ata_wdc.c: revision 1.91
cpu_intr_p() doesn't account for software interrupts (e.g. callouts) so
we can't use it here. Rssurect ATACH_TH_RUN, backing out
src/sys/dev/ata/ata.c 1.101
src/sys/dev/ata/ata_wdc.c 1.90
src/sys/dev/ata/atavar.h 1.77
src/sys/dev/ic/wdc.c 1.255
src/sys/dev/scsipi/atapi_wdc.c 1.108
Should fix kern/39927 and kern/39725.
 1.101.2.2  28-Apr-2009  skrll Sync with HEAD.
 1.101.2.1  19-Jan-2009  skrll Sync with HEAD.
 1.102.4.1  13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.111.2.1  30-Apr-2010  uebayasi Sync with HEAD.
 1.112.2.3  31-May-2011  rmind sync with head
 1.112.2.2  21-Apr-2011  rmind sync with head
 1.112.2.1  30-May-2010  rmind sync with head
 1.113.2.1  06-Jun-2011  jruoho Sync with HEAD.
 1.115.8.1  29-Apr-2012  mrg sync to latest -current.
 1.115.4.4  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.115.4.3  16-Jan-2013  yamt sync with (a bit old) head
 1.115.4.2  30-Oct-2012  yamt sync with head
 1.115.4.1  17-Apr-2012  yamt sync with head
 1.125.2.4  03-Dec-2017  jdolecek update from HEAD
 1.125.2.3  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.125.2.2  23-Jun-2013  tls resync from head
 1.125.2.1  20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.127.4.2  18-May-2014  rmind sync with head
 1.127.4.1  28-Aug-2013  rmind sync with head
 1.130.2.1  10-Aug-2014  tls Rebase.
 1.132.8.39  29-Sep-2017  jdolecek introduce ATA_BSIZE and use it instead of DEV_BSIZE for get params and
recovery, where they are by spec 512 bytes and not negotiable
 1.132.8.38  27-Sep-2017  jdolecek restore the atac_claim_hw and atac_free_hw hooks, they are used on atari
 1.132.8.37  25-Sep-2017  jdolecek put recovery command on head of active xfers queue, so that drivers using
ata_queue_get_active_xfer() like mvsata(4) will find it and not random other
NCQ xfer

never return NCQ xfer from ata_queue_get_active_xfer(), if the first xfer
is NCQ simply return NULL

adjust ata_timo_xfer_check() to not check the expiring flag in C_WAITTIMO
branch, if we get there we ought to get the timeout handler aborted
 1.132.8.36  23-Sep-2017  jdolecek note to previous: the KASSERT() insisted actually there is at most one
active xfer, which is false during NCQ error recovery
 1.132.8.35  23-Sep-2017  jdolecek remove ata_queue_get_active_xfer() KASSERT() about having at least one
active xfer; it can happen we get interrupt while no longer having the cmd
active e.g. during recovery, and all callers handle getting NULL as result

fixes panic in mvsata(4) during error recovery
 1.132.8.34  20-Sep-2017  jdolecek add forgotten destroy of queue_idle cv in ata_queue_free()
 1.132.8.33  19-Sep-2017  jdolecek replace all remaining tsleep()/wakeup() calls with condition variables, or
calls to ata_delay(), as appropriate; change ata_delay() to require the
channel lock on entry, and pass the lock to kpause() for unlocking while
sleeping
 1.132.8.32  11-Sep-2017  jdolecek move debug printf where it belongs
 1.132.8.31  10-Sep-2017  jdolecek refactor code so that xfer c_start() hook is called with channel mutex held,
and hence the controller submit code no longer relies on spl

tested all the affected drivers - wdc (via piixide), ahci, mvsata, siisata,
both disk and atapi I/O
 1.132.8.30  10-Sep-2017  jdolecek fix regression in atabus_thread() when it was converted from spl to mutex -
the reset and c_start() routines expect to run on splbio; wrap the calls
insite splbio/splx() again for now, since we can't hold the mutex while calling
them

fixes problem experienced by Jonathan, where drive setup triggered
an spurious interrupt and panic due to state < READY
 1.132.8.29  15-Aug-2017  jdolecek explicitly do not try to activate any further commands when running recovery
xfer; it was kind of implied since the code would not queue another non-NCQ
command when non-NCQ command is active, but this is better for readibility
 1.132.8.28  12-Aug-2017  jdolecek use AT_LBA48 flag for the READ LOG EXT - it's required so that e.g. mvsata()
executes the command using wdccommandext(), it fails when executed using
wdccommand()
 1.132.8.27  12-Aug-2017  jdolecek take atabus_qlock while inserting into atabus_initq_head to avoid race
in attach and rescan; just cleanup, noticed this while doing the thread
locking, don't think there is realistic way to trigger this
 1.132.8.26  12-Aug-2017  jdolecek convert the atabus thread to use the channel lock and a condvar, adjust
code which sets the relevant channel flags to take the lock while doing so
 1.132.8.25  12-Aug-2017  jdolecek remove all logic around ATACH_IRQ_WAIT and channel-global ch_error/ch_status,
so that there is less hidden state shared by commands; primary intent is
to make the NCQ and non-NCQ paths more similar, and remove possibility
of incorrect handling for the NCQ commands

tested both disk and ATAPI - piixide(4) on QEMU, and siisata(4),
ahcisata(4), mvsata(4) on real hw
 1.132.8.24  01-Aug-2017  jdolecek make atastart() schedule as many commands as possible, instead of always
only one; makes it able to pick up pace again after processing non-NCQ command
 1.132.8.23  01-Aug-2017  jdolecek move the drive recovery block to drive struct, it's inherently per-drive
 1.132.8.22  29-Jul-2017  jdolecek reserve the highest slot for error recovery, and also have ata_channel
include space for the READ LOG EXT sector, so that it's not necessary
to allocate memory on the error handling path; now ata_read_log_ext_ncq()
will never fail due to resource shortage
 1.132.8.21  22-Jul-2017  jdolecek provide channel flag when executing NCQ commands, so that e.g. intr
handler can use this for handling decisions without checking xfer
 1.132.8.20  21-Jul-2017  jdolecek add checksum verification for data returned by READ LOG EXT; this is mostly
just paranoia for eventual driver/hw DMA bugs

this doesn't make difference for QEMU, as there the command actually always
just fails (it's not implemented)
 1.132.8.19  19-Jul-2017  jdolecek tighen and expand error handling, mostly for NCQ use cases:
- make retry timeout callout per xfer, i.e. retry separately
- zero whole bio struct on retry to avoid more stale state
- add a REQUEUE option, which doesn't bump retry count
- add ata_read_log_ext_ncq() for NCQ recovery
- adjust logic for activating xfers - allow next command only when
it's for same drive, several concurrent are only supported when HBA
and driver support FIS-based switching
- add new ata_timeout() which handles race between callout_stop()
and the invokation, add appropriate handling on deactivate/free paths
- stop using ch_status/ch_error in non-wdc code; later it will be dropped
completely
 1.132.8.18  27-Jun-2017  jdolecek attend error paths, more strict asserts and code consistency

- atastart() and ata_kill_pending() now KASSERT() that all xfers on queue
have same channel
- inactive xfers are killed via new reason KILL_GONE_INACTIVE, controller
code must not call any resource deactivation in that case
- c_intr() must call ata_waitdrain_xfer_check() as first thing, and must not
further touch any xfer structures on exit path; any resource cleanup
is supposed to be done in c_kill_xfer()
- c_kill_xfer() should never call atastart()
- ata_waitdrain_check() removed, replaced by ata_waitdrain_xfer_check()
- ATA_DRIVE_WAITDRAIN handling converted to use condvar
- removed unused ata_c callback
 1.132.8.17  24-Jun-2017  jdolecek change ata_delay() to tsleep for 1 hz rather than indefinitely if
provided ms is lower than 1 hz
 1.132.8.16  23-Jun-2017  jdolecek restart I/O processing after freeing xfer, i.e. now even after commands
like cache flush or standby; the command handling no longer use on-stack xfer,
hence use queue slot and compete with normal I/O for the xfers

the restart give change to all drives attached to the same channel
in round-robin fashion, for fair usage and to recover from situation
when disk is idle due to all xfers being consumed by other drives

make special concession for flush cache - ignore any new I/O requests
on the particular disk while the flush cache is waiting for xfer,
so that I/O queue won't starve the flush cache and the flush cache would
be done ASAP

tested on piixide(4), ahci(4), siisata(4)
 1.132.8.15  21-Jun-2017  jdolecek hold channel lock for ata_exec_xfer() and most of atastart(), convert
C_WAITACT handling to use condvar

add comment for the code block with C_FREE and ata_free_xfer() explaining
why it's not abstraction layer violation
 1.132.8.14  21-Jun-2017  jdolecek two last forgotten on-stack xfers replaced with using the queue xfer -
ata_get_params() and ata_set_mode()

fix wdc_atapi_get_params() to free the xfer used for soft reset before
calling ata_get_params() - it's now necessary, as ata_get_params() doesn't
invent a private xfer own any more
 1.132.8.13  21-Jun-2017  jdolecek change ata_queue_hwslot_to_xfer() and ata_queue_get_active_xfer() to take
ata_channel instead of ata_queue as parameter, and lock the channel while
traversing the queue
 1.132.8.12  20-Jun-2017  jdolecek remove ata queue downsizing - every device, attached to the same channel,
uses slots according to it's own limits

wdc code changed to expect maximum one active xfer, and not check number
of openings in the channel; this is to facilitate using wdc functions
for e.g. handling of atapi commands for drivers which support both ATAPI
and NCQ
 1.132.8.11  19-Jun-2017  jdolecek add ata_channel lock, use it to protect queue manipulation (only that for now);
add ata_channel_detach() to destroy the locks

change ata_get_xfer() so that it can wait for xfer, convert all on-stack
xfer code to use the blocking variant

fix siisata_reset_drive() to use polled reset and not try ata_activate_xfer(),
convert drive probe code also over from slot0 XXX to ata_get_xfer()

drive reset and PMP now works on siisata(4) too; changes tested also
on piixide(4), ahci(4), mvsata(4)
 1.132.8.10  17-Jun-2017  jdolecek make PMP working great again

tested with mvsata(4), my ahcisata(4) controller unfortunately doesn't
support PMP
 1.132.8.9  16-Jun-2017  jdolecek adjust reset channel and dump paths
- channel reset now always kills active transfer, even on dump path, but
now doesn't touch the queued waiting transfers; also kill_xfer hook is
always called, so that HBA can free any private xfer resources and thus
the dump request has chance to work
- kill_xfer routines now always call ata_deactivate_xfer(); added KASSERT()s
to ata_free_xfer() to expect deactivated xfer
- when called during channel reset before dump, ata_kill_active() drops
any queued waiting transfers without processing
- do not (re)queue any transfers in wddone() when dumping
- kill AT_RST_NOCMD flag

This should also hopefully fix the 'polled command has been queued' panic
as reported in:
PR kern/11811 by John Hawkinson
PR kern/47041 by Taylor R Campbell
PR kern/51979 by Martin Husemann

dump tested working with piixide(4) and ahci(4). mvsata(4) dump times out,
but otherwise tested working, will be fixed separately. siisata(4) mechanically
changed and not tested.
 1.132.8.8  24-Apr-2017  jdolecek set NCQ priority field to 'high' for BPRIO_TIMECRITICAL transfers if drive
supports it
 1.132.8.7  23-Apr-2017  jakllsch Use WDC_QUEUE_DEPTH_MASK to correctly mask the NCQ depth IDENTIFY data.
 1.132.8.6  19-Apr-2017  jdolecek add ATA FUA support
 1.132.8.5  19-Apr-2017  jdolecek adjust ata code to support more than one active command, including the
timeout handling, add support for NCQ commands

move probe for NCQ and number of tags to middle layer, negotiate mutual
support between drive and controller

implement NCQ support in ahci(4)
 1.132.8.4  15-Apr-2017  jdolecek pass also ata_command via ata_xfer, callers of ata_exec_command() is now
responsible for allocation/disposal of the structure

change code to allocate ata_xfer for commands on stack same way as previously
the ata_command were, using c_slot 0; adjust asserts so that it would allow
several xfers with same c_slot, as long as only one such transfer is active
at a time
 1.132.8.3  15-Apr-2017  jdolecek make ata_xfer's allocated as part of ata_queue and make it include ata_bio;
they are pre-allocated on attach and ata_get_xfer() now never sleep, drop the
pool

modify wd(4) to file the bio requests using the xfers and hence
make it possible to have more than one active I/O request in flight;
ata_bio callback doesn't need to allocate any memory any more,
require it to never return ATACMD_TRY_AGAIN

move lp, badsect, multi from ata_bio to ata_drive_datas, as they are per-drive,
not per transfer

drop unused drv atac_claim_hw/atac_free_hw hooks, and also drop again ata_bio
c_hwslot
 1.132.8.2  11-Apr-2017  jdolecek hide active_xfer from drivers
 1.132.8.1  10-Apr-2017  jdolecek ATA infrastructure improvements to eventually support more outstanding
commands

patch by Matt Thomas
 1.141.6.18  15-Oct-2018  jdolecek change the SATA/NCQ recovery to run in the atabus thread
 1.141.6.17  14-Oct-2018  jdolecek add comment to kthread_create() call to explain why atabus_thread is actually
running under KERNEL_LOCK()
 1.141.6.16  11-Oct-2018  jdolecek refactor shared parts of the SATA error recovery into new function
ata_recovery_resume() and use for ahcisata/siisata/mvsata, also replace
per-controller hold/unhold with generic version

move the shared recovery code into separate file ata_recovery.c
 1.141.6.15  06-Oct-2018  jdolecek actually, just make dump use the same queue skip as recovery, and remove the
no longer necessary ata_queue_reset() call from wd(4)

also for PR kern/47041
 1.141.6.14  06-Oct-2018  jdolecek remove AT_RST_EMERG, do the queue reset explicitly in wd(4)

this should explicitly fix PR kern/47041 with sync during heavy
disk activity, even thought it was actually already implicitly fixed by calling
ata_thread_run() for drive reset in previous commit already, since the
function already called ata_queue_reset()

drop now unused ch_reset_flags and drive_reset_flags
 1.141.6.13  06-Oct-2018  jdolecek remove extra newline
 1.141.6.12  06-Oct-2018  jdolecek fix ata_thread_run() for drive reset to set and check correct drive_flags
 1.141.6.11  03-Oct-2018  jdolecek change channel reset and drive reset for all ATA controllers to always
run via thread, and with channel lock held the whole time; the queue is
frozen while reset is pending

for this repurpose ata_reset_channel() into new ata_thread_run()

also adjust some device printfs to not leak xfer pointer, and avoid
aprint_* for non-autoconf messages
 1.141.6.10  24-Sep-2018  jdolecek replace KASSERT() for READ LOG EXT returning without error indication
with just log to avoid triggering panic under QEMU - it doesn't implement
the command for AHCI emulation
 1.141.6.9  22-Sep-2018  jdolecek fix use-after-free in wd(4) dump, detected by switch to the pool

change code in wd_dumpblocks() to use it's own non-pool ata_xfer,
which skips the deallocation step and thus keeps the contents when the I/O
is finished
 1.141.6.8  22-Sep-2018  jdolecek revert atabus_alloc_drives() to use KM_NOSLEEP again - it's called with
ch_lock held hence cannot actually sleep, fixing is out of scope of the branch
 1.141.6.7  22-Sep-2018  jdolecek use KM_SLEEP in atabus_alloc_drives() to avoid spurious failures; this is
not called from interrupt context
 1.141.6.6  22-Sep-2018  jdolecek separate ata_xfer slot allocation and the memory allocation, so that
there can be more queued xfers than number of supported slots by controller,
and use a pool instead of custom pre-allocation

primarily to help PR kern/52614

remove no longer needed custom wd(4) logic for flush cache

switch also wd(4) trim/suspend/setcache/wdioctlstrategy to sleep waiting
for the memory, they are all called from process context and this
avoids spurious failures
 1.141.6.5  17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.141.6.4  17-Sep-2018  jdolecek switch from TAILQ to SIMPLEQ for ata_xfer pending queue to save
space, don't need doubly linked queue
 1.141.6.3  17-Sep-2018  jdolecek move low-level protocol handlers hooks from ata_xfer to separate struct,
initialized statically

primarily to reduce ata_xfer struct size, but also improves readibility,
and enforces consistency
 1.141.6.2  01-Sep-2018  jdolecek stop assuming c_slot is immutable, use the xfer pointer rather than c_slot
in printfs()
 1.141.6.1  31-Aug-2018  jdolecek refactor ata_xfer to be just dumb structure; move all callouts/condvars out

retry callout to wd(4); reset callout and the active/cmd finish condvars
to channel queue; change code using the condvars so it works if there
are multiple waiters

simplify the async wait code for cmds, replace ata_wait_xfer()/ata_wake_xfer()
with ata_wait_cmd()

fix the callout_invoking/ack race handling code for timeouts to
actually have chance to work; change mvsata(4) to use generic timeout func

towards resolution of kern/52614
 1.141.4.4  21-Apr-2020  martin Sync with HEAD
 1.141.4.3  13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.141.4.2  08-Apr-2020  martin Merge changes from current as of 20200406
 1.141.4.1  10-Jun-2019  christos Sync with HEAD
 1.141.2.2  26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.141.2.1  26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.149.2.3  30-Dec-2022  martin Pull up following revision(s) (requested by tsutsui in ticket #1557):

sys/dev/ic/ahcisata_core.c: revision 1.83
sys/dev/ic/ahcisata_core.c: revision 1.102
sys/dev/ata/ata.c: revision 1.164
sys/dev/ata/ata_wdc.c: revision 1.115
sys/dev/ata/ata_recovery.c: revision 1.4
sys/dev/ic/siisata.c: revision 1.42
sys/dev/ic/wdc.c: revision 1.308
sys/dev/ic/mvsata.c: revision 1.56
sys/dev/scsipi/atapi_wdc.c: revision 1.138
sys/dev/ic/siisata.c: revision 1.49
sys/dev/ata/atavar.h: revision 1.105
sys/dev/ata/wd.c: revision 1.460
sys/dev/ata/ata.c: revision 1.155
sys/dev/ata/wd.c: revision 1.462
sys/dev/ata/atavar.h: revision 1.109
sys/dev/ata/satapmp_subr.c: revision 1.16
sys/dev/ic/wdc.c: revision 1.299
sys/dev/ic/ahcisata_core.c: revision 1.93
sys/dev/ata/ata_wdc.c: revision 1.120
sys/dev/ic/wdcvar.h: revision 1.100
sys/dev/scsipi/atapi_wdc.c: revision 1.141
sys/dev/ic/mvsata.c: revision 1.61
sys/dev/usb/umass_isdata.c (apply patch)

drop wd lock in wdstart1() before calling the ata_bio hook; when called
from ata thread context, that can still need to sleep for wdc attachments
in wdcwait()

fix use-after-free for ata xfer on bio submission found by KASAN
driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself
PR kern/55169 by Nick Hudson

Function declaration formating whitespace consistency. NFCI.

PR kern/56403
Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.
(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.
Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().
(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).
"Go ahead" by jdolecek@.
 1.149.2.2  30-Aug-2022  martin Pull up following revision(s) (requested by tsutsui in ticket #1512):

sys/dev/ata/ata.c: revision 1.163

Output missing '\n' for capability list when DMA support is not compiled in.
 1.149.2.1  15-Mar-2022  martin Pull up following revision(s) (requested by perseant in ticket #1435):

sys/dev/ata/ata.c: revision 1.167

Avoid an unaccounted extra channel freeze, if a reset is requested
more than once before the thread services the request. Closes PR#56745.
 1.154.2.2  25-Apr-2020  bouyer Sync with bouyer-xenpvh-base2 (HEAD)
 1.154.2.1  20-Apr-2020  bouyer Sync with HEAD
 1.160.4.1  02-Apr-2021  thorpej config_found_ia() -> config_found() w/ CFARG_IATTR.
 1.161.8.1  04-Aug-2021  thorpej Adapt to CFARGS().

RSS XML Feed