Home | History | Annotate | only in /src/sys/dev/ata
History log of /src/sys/dev/ata
RevisionDateAuthorComments
 1.6 11-Dec-2005  christos merge ktrace-lwp.
 1.5 14-Dec-2003  thorpej Serial ATA register definitions.
 1.4 05-Aug-2002  soren branches: 1.4.6;
Add ATA SMART reporting support from Ben Collver in kern/12787.
 1.3 11-Apr-2001  jdolecek branches: 1.3.2; 1.3.14;
Put back all header files defining ioctls
 1.2 12-Oct-1998  bouyer branches: 1.2.12; 1.2.24;
Adjust include list.
 1.1 12-Jun-1998  cgd Rework the way kernel include files are installed. In the new method,
as with user-land programs, include files are installed by each directory
in the tree that has includes to install. (This allows more flexibility
as to what gets installed, makes 'partial installs' easier, and gives us
more options as to which machines' includes get installed at any given
time.) The old SYS_INCLUDES={symlinks,copies} behaviours are _both_
still supported, though at least one bug in the 'symlinks' case is
fixed by this change. Include files can't be build before installation,
so directories that have includes as targets (e.g. dev/pci) have to move
those targets into a different Makefile.
 1.2.24.2 13-Aug-2002  nathanw Catch up to -current.
 1.2.24.1 21-Jun-2001  nathanw Catch up to -current.
 1.2.12.1 21-Apr-2001  bouyer Sync with HEAD
 1.3.14.1 29-Aug-2002  gehenna catch up with -current.
 1.3.2.1 06-Sep-2002  jdolecek sync kqueue branch with HEAD
 1.4.6.3 21-Sep-2004  skrll Fix the sync with head I botched.
 1.4.6.2 18-Sep-2004  skrll Sync with HEAD.
 1.4.6.1 03-Aug-2004  skrll Sync with HEAD
 1.13 24-Oct-2018  jdolecek remove TODO.ncq; the FIS note was added to atastart(), and the note
about dump to unopened wd(4) device was attached to PR kern/16789, and
nothing else remains
 1.12 24-Oct-2018  jdolecek move the comment (and expand) about NCQ TRIM to wd_trim()
 1.11 24-Oct-2018  jdolecek detach the controller itself on shutdown; adjust to not detach already
detached atabus/channel
 1.10 23-Oct-2018  jdolecek add support for DIOCCACHESYNC (!), and DIOCGCACHE; code adapted from ccd(4)

tested with Intel ATA Raid
 1.9 22-Oct-2018  jdolecek remove the note for dump, moral equivalent is done and biodone() most
certainly should not be done
 1.8 22-Oct-2018  jdolecek remove the wd* at umass? entry (comment moved to umass_isdata.c and manpage)

remove MSI for ahcisata(4), siisata(4) - these were done
 1.7 22-Oct-2018  jdolecek move mvsata(4) related items from ata/TODO.ncq to dev/ic/mvsata.c, it's
unlikely this will ever be done, but still good to note somewhere
 1.6 22-Oct-2018  jdolecek Merge jdolecek-ncqfixes branch

- ata_xfer's are dynamicall allocated as needed using a pool, no longer
limited to number of possible openings supported by controller; dump
and recovery paths use dedicated pre-allocated storage
- moved callouts and condvars from ata_xfer to queue or channel, so that
ata_xfer does not need special initialization
- slot allocation now done when xfer is being activated, uncoupled
from memory allocation; active slots are no longer tracked by controller
code
- channel and drive reset is done always via the atabus thread, and
now executes with channel locked the whole time
- NCQ recovery moved to shared function, and run via the thread also
- added some workarounds for buggy error recovery AHCI emulation in QEMU
and Parallels

designed to primarily fix kern/52614, but might also help with kern/47041
and kern/53183
 1.5 14-Oct-2018  jdolecek remove M_CANFAIL flag for malloc(9) - it was completely ignored, so had
actually no effect
 1.4 31-Aug-2018  jdolecek branches: 1.4.2;
one less
 1.3 01-Nov-2017  jdolecek branches: 1.3.2; 1.3.4; 1.3.6;
one down - mlelstv converted wd(4) to dksubr
 1.2 07-Oct-2017  jdolecek Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.1 19-Apr-2017  jdolecek branches: 1.1.2;
file TODO.ncq was initially added on branch jdolecek-ncq.
 1.1.2.47 29-Sep-2017  jdolecek introduce ATA_BSIZE and use it instead of DEV_BSIZE for get params and
recovery, where they are by spec 512 bytes and not negotiable
 1.1.2.46 28-Sep-2017  jdolecek add sysctls to control if NCQ is being used, and how many max tags;
I have a drive which is significantly slower with NCQ than non-NCQ,
and it's generally useful to have this easily overridable

while here, also move the frequency settings for WD_CHAOS_MONKEY
to a sysctl and make it per-drive
 1.1.2.45 28-Sep-2017  jdolecek add note about DEV_BSIZE use
 1.1.2.44 21-Sep-2017  jdolecek wddone() doesn't need more locking
 1.1.2.43 20-Sep-2017  jdolecek MVSATA_WITHOUTDMA seems to work fine
 1.1.2.42 19-Sep-2017  jdolecek replace all remaining tsleep()/wakeup() calls with condition variables, or
calls to ata_delay(), as appropriate; change ata_delay() to require the
channel lock on entry, and pass the lock to kpause() for unlocking while
sleeping
 1.1.2.41 13-Sep-2017  jdolecek two more - mvsata MVSATA_WITHOUTDMA seems to be broken, and latest channel lock
changes introduced panic in mi_switch() on code paths which kpause/tsleep
 1.1.2.40 11-Sep-2017  jdolecek doesn't seem the freeze/thaw in error recovery can cause the thread to panic,
all commands within are executed as polled and hence thread is not invoked
 1.1.2.39 11-Sep-2017  jdolecek atastart() rechecked - it's okay to call it even in error path in those
several cases we do, it's just optimization to skip the call; I'm not even
very convinced it's useful to have this conditional, but keeping for now
until proven harmful
 1.1.2.38 10-Sep-2017  jdolecek refactor code so that xfer c_start() hook is called with channel mutex held,
and hence the controller submit code no longer relies on spl

tested all the affected drivers - wdc (via piixide), ahci, mvsata, siisata,
both disk and atapi I/O
 1.1.2.37 29-Aug-2017  jdolecek some notes around c_start/atastart()
 1.1.2.36 13-Aug-2017  jdolecek move the dump note to post-merge, it's not new
siisata seems to be fine, no longer holds the merge

remove the kill active transfers after software drive reset - not relevant

now only the wd* at umass? stays as new item, but I won't hold the merge
for this, as I don't have the hardware and it is contained enough to be
resolved on HEAD
 1.1.2.35 12-Aug-2017  jdolecek add NCQ error recovery for mvsata(4)

fix wait flags for mvsata_bio_start(), AT_* constants are only valid for cmd

change the debug logging to similar form as ahcisata/siisata, so it's
easier to selectively show
 1.1.2.34 12-Aug-2017  jdolecek convert the atabus thread to use the channel lock and a condvar, adjust
code which sets the relevant channel flags to take the lock while doing so
 1.1.2.33 12-Aug-2017  jdolecek remove all logic around ATACH_IRQ_WAIT and channel-global ch_error/ch_status,
so that there is less hidden state shared by commands; primary intent is
to make the NCQ and non-NCQ paths more similar, and remove possibility
of incorrect handling for the NCQ commands

tested both disk and ATAPI - piixide(4) on QEMU, and siisata(4),
ahcisata(4), mvsata(4) on real hw
 1.1.2.32 01-Aug-2017  jdolecek another one down
 1.1.2.31 31-Jul-2017  jdolecek some more notes
 1.1.2.30 19-Jul-2017  jdolecek update to note remaining work

move some stuff to 'after-merge'
 1.1.2.29 03-Jul-2017  jdolecek expand what needs to be done with error handling
 1.1.2.28 28-Jun-2017  jdolecek ATAPI on siisata(4), ahcisata(4) tested
 1.1.2.27 27-Jun-2017  jdolecek note ATAPI on siisata, ahcisata needs to be tested
 1.1.2.26 24-Jun-2017  jdolecek fix dump on mvsata - there was misplaced mvsata_bio_intr() call causing
diagnostic panic, and we also need to switch controller to appropriate
DMA mode (NCQ vs. non-NCQ DMA)

while here, also reduce delay for the polled command wait to 100 us, same
as siisata(4) and ahcisata(4)
 1.1.2.25 24-Jun-2017  jdolecek add detach code for umass_isdata; compile-tested only
 1.1.2.24 24-Jun-2017  jdolecek only limit the openings for I/O xfer if the drive actually supports NCQ; if
it's non-NCQ drive, the tag is not going to be used, so we can use any xfer
 1.1.2.23 23-Jun-2017  jdolecek fix dump for siisata(4) to work - need to call drv_done() callback in order to
not leak xfers

while here, reduce the DELAY() for polled bio to make it go much faster,
same value as used in ahcisata(4)

needs also rev. 1.252 src/sys/arch/x86/x86/pmap.c to not trigger the
assertion, the pmap.c fix committed only on HEAD
 1.1.2.22 23-Jun-2017  jdolecek restart I/O processing after freeing xfer, i.e. now even after commands
like cache flush or standby; the command handling no longer use on-stack xfer,
hence use queue slot and compete with normal I/O for the xfers

the restart give change to all drives attached to the same channel
in round-robin fashion, for fair usage and to recover from situation
when disk is idle due to all xfers being consumed by other drives

make special concession for flush cache - ignore any new I/O requests
on the particular disk while the flush cache is waiting for xfer,
so that I/O queue won't starve the flush cache and the flush cache would
be done ASAP

tested on piixide(4), ahci(4), siisata(4)
 1.1.2.21 21-Jun-2017  jdolecek hold channel lock for ata_exec_xfer() and most of atastart(), convert
C_WAITACT handling to use condvar

add comment for the code block with C_FREE and ata_free_xfer() explaining
why it's not abstraction layer violation
 1.1.2.20 20-Jun-2017  jdolecek some more notes for dump on unopened device (PR kern/16789)
 1.1.2.19 20-Jun-2017  jdolecek remove ata queue downsizing - every device, attached to the same channel,
uses slots according to it's own limits

wdc code changed to expect maximum one active xfer, and not check number
of openings in the channel; this is to facilitate using wdc functions
for e.g. handling of atapi commands for drivers which support both ATAPI
and NCQ
 1.1.2.18 19-Jun-2017  jdolecek add ata_channel lock, use it to protect queue manipulation (only that for now);
add ata_channel_detach() to destroy the locks

change ata_get_xfer() so that it can wait for xfer, convert all on-stack
xfer code to use the blocking variant

fix siisata_reset_drive() to use polled reset and not try ata_activate_xfer(),
convert drive probe code also over from slot0 XXX to ata_get_xfer()

drive reset and PMP now works on siisata(4) too; changes tested also
on piixide(4), ahci(4), mvsata(4)
 1.1.2.17 17-Jun-2017  jdolecek more notes, esp. PMP on siisata doesn't work + more problems with downsize
 1.1.2.16 17-Jun-2017  jdolecek make PMP working great again

tested with mvsata(4), my ahcisata(4) controller unfortunately doesn't
support PMP
 1.1.2.15 16-Jun-2017  jdolecek notes from dump testing - TL;DR: broken, fixing
 1.1.2.14 13-Jun-2017  jakllsch Add NCQ support to siisata(4).

There are still issues that need to be address before I consider this
mergable.
 1.1.2.13 12-Jun-2017  jakllsch Add a few more NCQ branch action items.
 1.1.2.12 12-Jun-2017  jdolecek update notes
 1.1.2.11 24-Apr-2017  jdolecek set NCQ priority field to 'high' for BPRIO_TIMECRITICAL transfers if drive
supports it
 1.1.2.10 24-Apr-2017  jdolecek do not use PRD Byte Count for queued commands to determine number of bytes
actually transferred; it should not be used and is not guaranteed to be
valid according to AHCI spec, underflow is always invalid for queued cmds

this fixes my ls problem under QEMU
 1.1.2.9 24-Apr-2017  jdolecek one down
 1.1.2.8 23-Apr-2017  jdolecek note possible problem with ata_exec_xfer()
 1.1.2.7 21-Apr-2017  jdolecek move the wd* at umass? item to bugs, too
 1.1.2.6 21-Apr-2017  jdolecek group items, slightly expand description
 1.1.2.5 20-Apr-2017  jdolecek remove ahci timeouts, fixed by Jonathan
 1.1.2.4 20-Apr-2017  jdolecek use a fake ata_channel struct in umass_isdata.c so that the wd(4) calls
to ata_{get,free}_xfer() can work, adjust to ata interface changes

compile tested only
 1.1.2.3 19-Apr-2017  jdolecek note the QEMU bug
 1.1.2.2 19-Apr-2017  jdolecek up to 65536 sectors can be transferred for LBA48 commands; adjust wdminphys()
and wdstart1() accordingly
 1.1.2.1 19-Apr-2017  jdolecek bunch of random observations
 1.3.6.1 10-Jun-2019  christos Sync with HEAD
 1.3.4.3 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.3.4.2 20-Oct-2018  pgoyette Sync with head
 1.3.4.1 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.3.2.2 03-Dec-2017  jdolecek update from HEAD
 1.3.2.1 01-Nov-2017  jdolecek file TODO.ncq was added on branch tls-maxphys on 2017-12-03 11:36:59 +0000
 1.4.2.17 21-Oct-2018  jdolecek testing done (including PMP on siisata(4) and ATAPI cd(4) on ahcisata/siisata)
 1.4.2.16 17-Oct-2018  jdolecek rechecked bad block handling under Parallels and it really doesn't set
any error bits in any of the registers (neither IS nor SERR) for out-of-range
block read nor write, while it works just fine under VMware, so not our bug
 1.4.2.15 15-Oct-2018  jdolecek change the SATA/NCQ recovery to run in the atabus thread
 1.4.2.14 14-Oct-2018  jdolecek adjust mvsata_bio_intr() so it recognizes the 'tfd' parameter as passed
by recovery and hence works; use it also for passing state from
mvsata_edma_handle()
 1.4.2.13 11-Oct-2018  jdolecek refactor shared parts of the SATA error recovery into new function
ata_recovery_resume() and use for ahcisata/siisata/mvsata, also replace
per-controller hold/unhold with generic version

move the shared recovery code into separate file ata_recovery.c
 1.4.2.12 06-Oct-2018  jdolecek move the entry for error recovery on thread up, it will be done on ncqfixes
branch
 1.4.2.11 04-Oct-2018  jdolecek relax the NOERROR + C_CHAOS, just rerun the xfer instead of panic, to handle
another variant of bad/missing AHCI error recovery, this time under Parallels
 1.4.2.10 03-Oct-2018  jdolecek change channel reset and drive reset for all ATA controllers to always
run via thread, and with channel lock held the whole time; the queue is
frozen while reset is pending

for this repurpose ata_reset_channel() into new ata_thread_run()

also adjust some device printfs to not leak xfer pointer, and avoid
aprint_* for non-autoconf messages
 1.4.2.9 24-Sep-2018  jdolecek replace KASSERT() for READ LOG EXT returning without error indication
with just log to avoid triggering panic under QEMU - it doesn't implement
the command for AHCI emulation
 1.4.2.8 22-Sep-2018  jdolecek one more
 1.4.2.7 22-Sep-2018  jdolecek remove explicit ata_channel_start() calls, it's no longer necessary
now that ata_xfer's are allocated via pool and not really limited;
replace by just a callout to restart the processing for rare cases
where system runs out of memory
 1.4.2.6 22-Sep-2018  jdolecek fix use-after-free in wd(4) dump, detected by switch to the pool

change code in wd_dumpblocks() to use it's own non-pool ata_xfer,
which skips the deallocation step and thus keeps the contents when the I/O
is finished
 1.4.2.5 22-Sep-2018  jdolecek use KM_SLEEP in atabus_alloc_drives() to avoid spurious failures; this is
not called from interrupt context
 1.4.2.4 22-Sep-2018  jdolecek separate ata_xfer slot allocation and the memory allocation, so that
there can be more queued xfers than number of supported slots by controller,
and use a pool instead of custom pre-allocation

primarily to help PR kern/52614

remove no longer needed custom wd(4) logic for flush cache

switch also wd(4) trim/suspend/setcache/wdioctlstrategy to sleep waiting
for the memory, they are all called from process context and this
avoids spurious failures
 1.4.2.3 17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.4.2.2 31-Aug-2018  jdolecek fix comment - dynamically allocated yes, per-drive not
 1.4.2.1 31-Aug-2018  jdolecek list goals for the jdolecek-ncqfixes branch
 1.171 17-Feb-2025  jakllsch Restore ATA DMA mode downgrade support everywhere; it's a necessary part
of any system supporting (parallel) ATA DMA. There is hardware out there,
including cobalt, macppc, and sparc64 where this functionality is
necessary to avoid non-functional disks, either in as-shipped hardware
configurations or with add-in cards, or perhaps just with compromised
IDE/PATA cables.

Should address:
PR 58767
PR 59023
PR 59078

If anyone really insists on not having this support they can now turn it
off themselves with `options ATA_NO_DOWNGRADE_MODE`
 1.170 17-Feb-2025  jakllsch Fix aprint_error_dev() abuse
 1.169 31-May-2022  andvar branches: 1.169.10;
fix various typos in comments, documentation and messages.
 1.168 28-May-2022  andvar fix various typos in comments.
 1.167 14-Mar-2022  perseant Avoid an unaccounted extra channel freeze, if a reset is requested
more than once before the thread services the request. Closes PR#56745.
 1.166 23-Feb-2022  andvar fix various typos in comments, mainly immediatly/immediately/,
as well shared and recently fixed typos in OpenBSD code by Jonathan Grey.
 1.165 12-Nov-2021  skrll Trailing whitespace
 1.164 05-Oct-2021  rin PR kern/56403

Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.

(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.

Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().

(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).

"Go ahead" by jdolecek@.
 1.163 29-Aug-2021  rin Output missing '\n' for capability list when DMA support is not compiled in.
 1.162 07-Aug-2021  thorpej Merge thorpej-cfargs2.
 1.161 24-Apr-2021  thorpej branches: 1.161.8;
Merge thorpej-cfargs branch:

Simplify and make extensible the config_search() / config_found() /
config_attach() interfaces: rather than having different variants for
which arguments you want pass along, just have a single call that
takes a variadic list of tag-value arguments.

Adjust all call sites:
- Simplify wherever possible; don't pass along arguments that aren't
actually needed.
- Don't be explicit about what interface attribute is attaching if
the device only has one. (More simplification.)
- Add a config_probe() function to be used in indirect configuiration
situations, making is visibly easier to see when indirect config is
in play, and allowing for future change in semantics. (As of now,
this is just a wrapper around config_match(), but that is an
implementation detail.)

Remove unnecessary or redundant interface attributes where they're not
needed.

There are currently 5 "cfargs" defined:
- CFARG_SUBMATCH (submatch function for direct config)
- CFARG_SEARCH (search function for indirect config)
- CFARG_IATTR (interface attribte)
- CFARG_LOCATORS (locators array)
- CFARG_DEVHANDLE (devhandle_t - wraps OFW, ACPI, etc. handles)

...and a sentinel value CFARG_EOL.

Add some extra sanity checking to ensure that interface attributes
aren't ambiguous.

Use CFARG_DEVHANDLE in MI FDT, OFW, and ACPI code, and macppc and shark
ports to associate those device handles with device_t instance. This
will trickle trough to more places over time (need back-end for pre-OFW
Sun OBP; any others?).
 1.160 03-Oct-2020  riastradh branches: 1.160.4;
autoconf: Blame devices holding up boot with config_pending.

Blame message requires `boot -x' (AB_DEBUG).

Fix ata so it doesn't mismatch config_pending_incr/decr devices.
 1.159 25-May-2020  jdolecek disable downgrade of ATA mode from DMA, as generally not relevant
any more - while it has been instrumental to inadvertedly discover
driver bugs in PIO mode under QEMU recently, generally the switch
more hurts than helps, so now only warn when DMA errors happen

code kept under ATA_DOWNGRADE_MODE ifdef, disabled by default
 1.158 25-May-2020  jdolecek make ata_downgrade_mode() static, it's not used anywhere else
 1.157 02-May-2020  thorpej Back out changes to use a threadpool for now; it's causing trouble
for some folks on Thinkpads.
 1.156 25-Apr-2020  thorpej Rather than creating a kthread-per-channel, use a threadpool and a
threadpool-job-per-channel for the in-thread-context work that needs
to be done (which is rare).

On one of my test systems, this results in the total number of LWPs
after multi-user boot dropping from 116 to 78.
 1.155 13-Apr-2020  jdolecek fix use-after-free for ata xfer on bio submission found by KASAN

driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself

PR kern/55169 by Nick Hudson
 1.154 04-Apr-2020  jdolecek branches: 1.154.2;
fix deadlock in wdcwait() when xfer timeout happens while the atabus
thread sleeps in wdcwait() - check current lwp rather than relying
on global ATACH_TH_RUN channel flag

should fix the hang part of the problem reported in
http://mail-index.netbsd.org/netbsd-users/2020/03/12/msg024249.html

thanks to Paul Ripke for providing extensive debugging info
 1.153 21-Oct-2019  christos chuq does not like insomniac allocations so unlock-alloc-lock instead.
 1.152 21-Oct-2019  christos Fix assert_sleepable() panic by allocating with NOSLEEP. The alternative is
to unlock and relock the channel, but seems more dangerous to do so.
 1.151 15-Oct-2019  chs convert more KM_NOSLEEP to KM_SLEEP and remove code to handle failures.
 1.150 21-Aug-2019  msaitoh Fix typo (s/controler/controller/).
 1.149 25-May-2019  christos branches: 1.149.2;
use PR_ZERO
 1.148 06-Apr-2019  uwe Make sure the aprint_verbose chain is terminated with a newline.
Check sep instead of printed, b/c the latter is reset several times.
 1.147 11-Dec-2018  jdolecek in atabus_detach(), shutdown the channel thread only after all the
children successfully detach; this is especially important for
atapibus, which needs the thread to execute commands

fixes hang on poweroff with root on ATAPI cd(4), reported by Christoph Badura
 1.146 12-Nov-2018  jdolecek hold channel lock during whole ata_dmaerr()/ata_downgrade_mode() -
according to code inspection this is safe, none of the set_modes
hooks execute anything which would be taking the lock

adresses PR kern/53714 by Andreas Gustafsson
 1.145 24-Oct-2018  jdolecek ata_kill_active() needs to drop channel drop before calling kill_xfer() hook,
otherwise code tries to lock against itself when deactivating the xfer
 1.144 24-Oct-2018  jdolecek add further note about FIS-based switching to atastart()
 1.143 24-Oct-2018  jdolecek detach the controller itself on shutdown; adjust to not detach already
detached atabus/channel
 1.142 22-Oct-2018  jdolecek Merge jdolecek-ncqfixes branch

- ata_xfer's are dynamicall allocated as needed using a pool, no longer
limited to number of possible openings supported by controller; dump
and recovery paths use dedicated pre-allocated storage
- moved callouts and condvars from ata_xfer to queue or channel, so that
ata_xfer does not need special initialization
- slot allocation now done when xfer is being activated, uncoupled
from memory allocation; active slots are no longer tracked by controller
code
- channel and drive reset is done always via the atabus thread, and
now executes with channel locked the whole time
- NCQ recovery moved to shared function, and run via the thread also
- added some workarounds for buggy error recovery AHCI emulation in QEMU
and Parallels

designed to primarily fix kern/52614, but might also help with kern/47041
and kern/53183
 1.141 28-Oct-2017  riastradh branches: 1.141.2; 1.141.4; 1.141.6;
Kill some more extern struct cfdriver declarations.

Down with externs in .c!
 1.140 20-Oct-2017  jdolecek move ata_queue_alloc(1) and ata_queue_free() calls to ata_channel_init()
and ata_channel_destroy() respectively, to make attachment code simpler,
and to make it easier to spot special queue manipulation like cmdide(4)

on topic of PR kern/52606
 1.139 19-Oct-2017  jdolecek more detailed debug info; also sync DEBUG_* values in wd.c with ata.c
 1.138 18-Oct-2017  jdolecek add newlines to the debug messages
 1.137 15-Oct-2017  jdolecek move the extra thaw for scheduled ata_reset_channel() to the function itself,
so it's done regardless if the actual reset is run from thread context
or e.g. call with AT_POLL; fixes a hang after xfer failure and dma downgrade

add some debugging printfs, so freeze/thaw issues are easier to track
 1.136 10-Oct-2017  jdolecek split off functions used by 'wd* at umass?' into separate file, unfortunately
the previous approach with NATABUS doesn't work for kernels which include
MODULAR, but not atabus - such as macppc and evbarm
 1.135 08-Oct-2017  jdolecek as a stopgap measure, include necessary parts of dev/ata/ata.c even
if no atabus is configured, so that kernels with just 'wd* at umass?'
and no atabus compile; fixes kernel builds for vax, pmax and evbmips
after NCQ branch merge

XXX wd* at umass? still actually untested after the merge, need to discuss
XXX what to do with it
 1.134 08-Oct-2017  mlelstv Don't assert channel lock in polled mode.

ahcisata (and maybe others) use ata_delay during initialization where
lock isn't taken.
 1.133 07-Oct-2017  jdolecek Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.132 10-Sep-2014  matt branches: 1.132.8;
g/u_int[0-9]*_t/ s/u_int/uint/g
 1.131 25-Jul-2014  dholland Add d_discard to all struct cdevsw instances I could find.

All have been set to "nodiscard"; some should get a real implementation.
 1.130 16-Mar-2014  dholland branches: 1.130.2;
Change (mostly mechanically) every cdevsw/bdevsw I can find to use
designated initializers.

I have not built every extant kernel so I have probably broken at
least one build; however I've also found and fixed some wrong
cdevsw/bdevsw entries so even if so I think we come out ahead.
 1.129 12-Oct-2013  christos Pass the device name in, so we can debug what deferred drivers did not work.
 1.128 07-Aug-2013  blymn Make sure a channel has drives attached before resetting it otherwise
wdc_reset will panic the kernel due to a null derefrence of ch_drives.
 1.127 03-Apr-2013  bouyer branches: 1.127.4;
Fix kernel dump on ahci controller, by making sure we won't sleep
while dumping:
- introduce ata_delay() which either use delay() or kpause()
depending on flags. use it in sata_reset_interface() and
some ahci functions
- kill ATA_NOSLEEP, it was tested but never set. use ATA_POLL instead.
- reduce delay while polling in ahci, to speed up the dump

Should fix PR kern/41095
 1.126 01-Nov-2012  abs Allow setting ATADEBUG_MASK to default value of atadebug_mask
 1.125 04-Aug-2012  bouyer branches: 1.125.2;
If ch_ndrives is > 0, then ch_drive is not supposed to be NULL.
 1.124 31-Jul-2012  bouyer Apply back changes that were reverted on Jul 24 and Jul 26 (general ata/wdc
cleanup and SATA PMP support), now that I'm back to fix the fallouts.
 1.123 29-Jul-2012  jakllsch Remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas. Originally part of a commit by bouyer@.
 1.122 26-Jul-2012  jakllsch Revert, with intention of restoring in a less invasive way, the SATA Port
Multiplier code.

ok christos@
 1.121 24-Jul-2012  jakllsch Revert dsl@'s changes of Sun, 15 Jul 2012 10:55:35 +0000 and
Sun, 15 Jul 2012 10:56:50 +0000, excepting the kernel version bump.
First step in reverting regressions to ata(4) subsystem during the addition of
port multiplier support.
 1.120 22-Jul-2012  jakllsch It turns out my previous commit here was off by one.
 1.119 15-Jul-2012  jakllsch Stopgap crash prevention when atadebug_mask includes DEBUG_PROBE bit.
 1.118 15-Jul-2012  dsl Some namespace protection (and add greppablity).
Prefix the DRIVE_ and DRIVET_ constants from atavar.h with ATA_.
Don't use an enum for drive_type - you don't know how big it will be.
Move driver_type to avoid implicit structure padding (esp on arm).
This change is purely lexical and mechanical.

Update to 6.99.9 - this wasn't done when the SATA PMP changes
were made - I'm sure they warranted a bump.
 1.117 02-Jul-2012  bouyer Add sata Port MultiPlier (PMP) support to the ata bus layer,
as described in
http://mail-index.netbsd.org/tech-kern/2012/06/23/msg013442.html
PMP support in integrated to the atabus layer.
struct ata_channel's ch_drive[] is not dynamically allocated, and ch_ndrive
(renamed to ch_ndrives) closely reflects the size of the ch_drive[] array.
Add helper functions atabus_alloc_drives() and atabus_free_drives()
to manage ch_drive[]/ch_ndrives.
Add wdc_maxdrives to struct wdc_softc so that bus front-end can specify
how much drive they really support (master/slave or single).
ata_reset_drive() callback gains a uint32_t *sigp argument which,
when not NULL, will contain the signature of the device being reset.
While there, some cosmetic changes:
- added a drive_type enum to ata_drive_datas, and stop encoding the
probed drive type in drive_flags (we were out of drive flags anyway).
- rename DRIVE_ATAPIST to DRIVE_ATAPIDSCW to better reflect what this
really is
- remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas
- factor out the interpretation of SATA signatures in sata_interpet_sig()

propagate these changes to the ATA HBA drivers, and add support for PMP
to ahcisata(4) and siisata(4).

Thanks to:
- Protocase (http://www.protocase.com/) which provided a system
with lots of controllers, SATA PMP and drive slots
- Conservation Genomics Laboratory, Department of Biology, New Mexico State
University for hosting the above system
- Brook Milligan, who set up remote access and has been very responsive
when SATA cable move was needed
 1.116 06-Apr-2012  isaki Fix a typo (that I found 5 years ago). Though the definition of
atabusiodetach_args is the same as atabusioscan_args (at the moment).
 1.115 30-Apr-2011  jakllsch branches: 1.115.4; 1.115.8;
Add simplistic atabus(4) rescan support.
 1.114 18-Apr-2011  rmind Replace few simple_lock and ltsleep/wakeup uses with mutex(9) and condvar(9).

Note to all: please replace old primitives in your code! Thanks.
 1.113 28-Mar-2010  snj branches: 1.113.2;
Spell "enough" properly.
 1.112 24-Feb-2010  dyoung branches: 1.112.2;
A pointer typedef entails trading too much flexibility to declare const
and non-const types, and the kernel uses both const and non-const
PMF qualifiers and device suspensors, so change the pmf_qual_t and
device_suspensor_t typedefs from "pointers to const" to non-pointer,
non-const types.
 1.111 21-Jan-2010  dyoung branches: 1.111.2;
Cosmetic: join some lines. Get out early on errors, change

switch (...) {
...
error = ...;
break;
}
return error;

to

switch (...) {
...
return ...;
}
 1.110 08-Jan-2010  dyoung Expand PMF_FN_* macros.
 1.109 12-Nov-2009  dyoung Remove superfluous activation hook.
 1.108 19-Oct-2009  bouyer Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !
 1.107 13-Sep-2009  pooka Wipe out the last vestiges of POOL_INIT with one swift stroke. In
most cases, use a proper constructor. For proplib, give a local
equivalent of POOL_INIT for the kernel object implementation. This
way the code structure can be preserved, and a local link set is
not hazardous anyway (unless proplib is split to several modules,
but that'll be the day).

tested by booting a kernel in qemu and compile-testing i386/ALL
 1.106 12-May-2009  cegger struct cfdata * -> cfdata_t, no functional changes intended.
 1.105 07-Apr-2009  dyoung Cosmetic: flag a potential problem in ata_channel_attach():
it initializes a callout that does not appear to be destroyed
anywhere. Every callout_init() should ordinarily be matched by a
callout_destroy().
 1.104 03-Apr-2009  dyoung Stop dereferencing a dangling device_t pointer and crashing: skip the
drives flagged DRIVE_ATAPI in atabus_activate(,DVACT_DEACTIVATE) just as
we skip them in atabus_detach() and in atabus_childdetched().

Make atabus_detach() parallel attachment more closely by calling
config_detach() on the child chp->ata_drives[i] instead of on
chp->ch_drive[i].drv_softc. Assert that ata_drives[i] and
ch_drive[i].drv_softc are equal, and set them both to NULL in
atabus_childdetached().
 1.103 02-Apr-2009  dyoung During shutdown, detach devices in an orderly fashion.

Call the detach routine for every device in the device tree, starting
with the leaves and moving toward the root, expecting that each
(pseudo-)device driver will use the opportunity to gracefully commit
outstandings transactions to the underlying (pseudo-)device and to
relinquish control of the hardware to the system BIOS.

Detaching devices is not suitable for every shutdown: in an emergency,
or if the system state is inconsistent, we should resort to a fast,
simple shutdown that uses only the pmf(9) shutdown hooks and the
(deprecated) shutdownhooks. For now, if the flag RB_NOSYNC is set in
boothowto, opt for the fast, simple shutdown.

Add a device flag, DVF_DETACH_SHUTDOWN, that indicates by its presence
that it is safe to detach a device during shutdown. Introduce macros
CFATTACH_DECL3() and CFATTACH_DECL3_NEW() for creating autoconf
attachments with default device flags. Add DVF_DETACH_SHUTDOWN
to configuration attachments for atabus(4), atw(4) at cardbus(4),
cardbus(4), cardslot(4), com(4) at isa(4), elanpar(4), elanpex(4),
elansc(4), gpio(4), npx(4) at isa(4), nsphyter(4), pci(4), pcib(4),
pcmcia(4), ppb(4), sip(4), wd(4), and wdc(4) at isa(4).

Add a device-detachment "reason" flag, DETACH_SHUTDOWN, that tells the
autoconf code and a device driver that the reason for detachment is
system shutdown.

Add a sysctl, kern.detachall, that tells the system to try to detach
every device at shutdown, regardless of any device's DVF_DETACH_SHUTDOWN
flag. The default for kern.detachall is 0. SET IT TO 1, PLEASE, TO
HELP TEST AND DEBUG DEVICE DETACHMENT AT SHUTDOWN.

This is a work in progress. In future work, I aim to treat
pseudo-devices more thoroughly, and to gracefully tear down a stack of
(pseudo-)disk drivers and filesystems, including cgd(4), vnd(4), and
raid(4) instances at shutdown.

Also commit some changes that are not easily untangled from the rest:

(1) begin to simplify device_t locking: rename struct pmf_private to
device_lock, and incorporate device_lock into struct device.

(2) #include <sys/device.h> in sys/pmf.h in order to get some
definitions that it needs. Stop unnecessarily #including <sys/device.h>
in sys/arch/x86/include/pic.h to keep the amd64, xen, and i386 releases
building.
 1.102 16-Nov-2008  bouyer branches: 1.102.4;
cpu_intr_p() doesn't account for software interrupts (e.g. callouts) so
we can't use it here. Rssurect ATACH_TH_RUN, backing out
src/sys/dev/ata/ata.c 1.101
src/sys/dev/ata/ata_wdc.c 1.90
src/sys/dev/ata/atavar.h 1.77
src/sys/dev/ic/wdc.c 1.255
src/sys/dev/scsipi/atapi_wdc.c 1.108
Should fix kern/39927 and kern/39725.
 1.101 02-Oct-2008  bouyer branches: 1.101.2; 1.101.4;
Kill ATACH_TH_RUN and use cpu_intr_p() instead.
 1.100 01-Oct-2008  bouyer Use a separate thread to probe/attach atabus's childrens. Fixes a deadlock
where the interrupt routine wants to wake up the atabus thread to perform a
reset, while the thread is blocked in wd's attach function.
 1.99 12-Jun-2008  cegger branches: 1.99.2;
use device_lookup_private to get softc
 1.98 18-Mar-2008  cube branches: 1.98.2; 1.98.4; 1.98.6; 1.98.8;
Split device_t and softc for ATA devices, as well as wd(4). Other
cosmetic changes where appropriate.
 1.97 29-Feb-2008  dyoung Use PMF_FN_ARGS, PMF_FN_PROTO.
 1.96 10-Jan-2008  dyoung branches: 1.96.2; 1.96.6;
Oops, remove some debug statements that slipped in there.
 1.95 10-Jan-2008  dyoung Use device_t, device_xname(), device_private(). Add atabus_childdetached()
to clean up pointers to detached children.
 1.94 04-Jan-2008  ad Start detangling lock.h from intr.h. This is likely to cause short term
breakage, but the mess of dependencies has been regularly breaking the
build recently anyhow.
 1.93 20-Dec-2007  pooka Don't allocate DEV_BSIZE from the stack, use kmem instead.
 1.92 09-Dec-2007  jmcneill branches: 1.92.2;
Merge jmcneill-pm branch.
 1.91 19-Oct-2007  ad branches: 1.91.2; 1.91.4; 1.91.6;
machine/{bus,cpu,intr}.h -> sys/{bus,cpu,intr}.h
 1.90 09-Jul-2007  ad branches: 1.90.6; 1.90.8; 1.90.12;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements
 1.89 01-Jul-2007  dsl Remove 'else' after 'return' for clarity
 1.88 17-Mar-2007  dyoung Let config(1) know that #define ATADEBUG goes in opt_ata.h. In
dev/ic/wdc.c and in dev/ata/ata.c, #include "opt_ata.h", and make
both the files compile with or *without* ATADEBUG. Do not compile
with ATADEBUG by default.
 1.87 12-Mar-2007  ad branches: 1.87.2; 1.87.4;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.
 1.86 04-Mar-2007  christos branches: 1.86.2;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
 1.85 09-Feb-2007  ad branches: 1.85.2;
Merge newlock2 to head.
 1.84 25-Dec-2006  wiz Spell "schedule" correctly. From Zafer Aydogan.
 1.83 16-Nov-2006  christos __unused removal on arguments; approved by core.
 1.82 25-Oct-2006  bouyer Add sata registers to struct wdc_regs. Add wdc_sataprobe(), a function
probing drives using the standard SATA registers; taken from various
PCI sata drivers. Export wdc_drvprobe() too.
 1.81 15-Oct-2006  itohy white space police
 1.80 14-Oct-2006  itohy Add ATAPI Pioneer (info from Linux) and ATA TDK (I have one)
as exceptions of string byte order.
Add string byte order shuffling for big-endian platforms.
 1.79 12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.78 30-Sep-2006  itohy A little effort against kernel bloat....
Exclude ATA DMA support if no ATA DMA capable drivers are compiled in.
 1.77 24-Sep-2006  jmcneill Add "name" parameter to powerhook_establish, to aid debugging. No objections
on tech-kern@
 1.76 07-Sep-2006  itohy branches: 1.76.2; 1.76.4;
Add PIOBM (busmastering transfer using ATA PIO mode) support.
The PIOBM is used by only one driver (will be added later,
stay tuned) and intruduce an attribute "ata_piobm" so that
it will be conditionally compiled in.
The "ata_dma" (busmastering transfer using ATA DMA mode) and
"ata_udma" (busmastering transfer using ATA Ultra DMA mode)
attributes are also added for consistency, but unused for now.
 1.75 27-Aug-2006  christos Add missing initializer
 1.74 29-Mar-2006  thorpej Use device_cfdata().
 1.73 22-Jan-2006  bouyer branches: 1.73.2; 1.73.4; 1.73.6; 1.73.8; 1.73.10;
For an IDENTIFY we have to wait for DRQ after issuing the command.
 1.72 11-Dec-2005  christos branches: 1.72.2;
merge ktrace-lwp.
 1.71 07-Jul-2005  drochner tighten the autoconf constraints by passing the atapi/ata/ata_hl attribute
to config_found()
(after some cleanup we might be able to kill SCSIPI_BUSTYPE_*)
 1.70 29-May-2005  christos branches: 1.70.2;
- remove casts [to avoid volatile]
- sprinkle const
- avoid shadowed variables
 1.69 20-May-2005  bouyer Close a window where (disks) interrupts could be enabled while the
ATACH_TH_RUN flag is still set. Should fix panic reported by Jukka Salmi
on netbsd-users@.
 1.68 16-May-2005  bouyer Add a ata_queue_idle() function, which freeze a queue and tsleep() until the
controller is idle.
Change the powerhook function to call ata_queue_idle() on standby/suspend,
and ata_reset_channel() on resume (to wake up the disks and start from a
clean state).
Fix PR kern/30194 by Lubomir Sedlacik.
 1.67 11-Apr-2005  matt Fix mode printing. (don't panic on unmatched devices)
 1.66 04-Mar-2005  tacha branches: 1.66.2;
Disestablish powerhook while detaching atabus.
 1.65 27-Feb-2005  perry nuke trailing whitespace
 1.64 26-Jan-2005  jmcneill Add powerhook for atabus.
 1.63 14-Nov-2004  soren branches: 1.63.4; 1.63.6;
s/using DMA data transfers/using DMA/ to make it fit in 80 columns.
DMA only applies to data transfers anyway..
 1.62 30-Oct-2004  bouyer Allow polled commands to be queued, if ata_exec_xfer() is called in thread
context. For this introduce 3 new xfer tags:
- C_WAIT, equivalent of AT_WAIT
- C_WAITACT, when there is a thread waiting on this xfer to become the active
one (that is, to be at the head of the queue)
- C_FREE, set by ata_free_xfer() when it can't free the xfer because it's
still in use. The holder should then free the xfer ASAP.

If ata_exec_xfer() is called with (C_POLL | C_WAIT), and there is already
xfers to be processed, assert C_WAITACT and sleep.
atastart() checks for C_WAITACT, and wakeup the thread waiting for this xfer
to become active if set. atastart() won't process this xfer, it's the
responsability of the thread waked up to handle it.

Fix (the right way) kern/27421 by Martin Husemann.
 1.61 28-Oct-2004  bouyer Call xfer->c_start() with the channel from the xfer, not the one from the
thread. They may be different if we have a shared queue.
 1.60 11-Sep-2004  bouyer If we're not going to handle a reset request because one if already pending,
decrease queue_freeze that we just increased (otherwise the queue stay
frozen after the reset).
Should fix kern/26873 and port-i386/26910.
 1.59 21-Aug-2004  thorpej Eliminate some gratuitous spl frobbing in atabus_thread().
 1.58 21-Aug-2004  thorpej atastart() (called only at splbio(), and from interrupts) can change
drive_flags, to make sure all drive_flags manipulations are done at
splbio().
 1.57 20-Aug-2004  thorpej - atabus_detach(): Manipulate ch_flags at splbio().
- ata_reset_channel(): Assert that we're at splbio() when called.
- Sprinkle some comments about functions that must be called at splbio().
 1.56 20-Aug-2004  thorpej WDC_DIAGNOSTIC -> ATA_DEBUG
 1.55 20-Aug-2004  thorpej - Add an (*ata_reset_channel)() member to ata_bustype.
- Add an ata_reset_channel() function that performs the common parts
of resetting an ATA channel, which uses the (*ata_reset_channel)()
callback to do the heavy lifting. Adjust callers to use ata_reset_channel()
instead of wdc_reset_channel().

This removes the last wdc-specific code from ata.c!
 1.54 20-Aug-2004  thorpej Use ANSI function decls for atabus open/close/ioctl.
 1.53 20-Aug-2004  thorpej wdcreg.h is not needed here any more.
 1.52 20-Aug-2004  thorpej Move common parts of attaching an ata_channel to a new ata_channel_attach()
function.
 1.51 20-Aug-2004  thorpej Move atabusconfig() to ata.c.
 1.50 20-Aug-2004  thorpej In ata_get_params(), use the (*ata_exec_command)() available from the
ata_bustype rather than using wdc_exec_command() directly.
 1.49 20-Aug-2004  thorpej Move most of wdc_softc into a new atac_softc structure that contains
info common to all types of ATA controllers.
 1.48 14-Aug-2004  thorpej - Split the register handles out of struct wdc_channel into a separate
wdc_regs structure, and array of which (indexed per channel) is pointed
to by struct wdc_softc.
- Move the resulting wdc_channel structure to atavar.h and rename it to
ata_channel. Rename the corresponding flags.
- Add a "ch_ndrive" member to struct ata_channel, which indicates the
maximum number of drives that can be present on the channel. For now,
this is always 2. Add an ATA_MAXDRIVES constant that places an upper
limit on this value, also currently 2.
 1.47 13-Aug-2004  thorpej WDCDEBUG -> ATADEBUG.
 1.46 13-Aug-2004  thorpej Don't bother with bits that tell of the presence of optional callbacks;
just check the function pointers for NULL.
 1.45 13-Aug-2004  thorpej Move wdcstart() to ata.c and rename it to atastart().
 1.44 13-Aug-2004  thorpej Move wdc_exec_xfer() to ata.c and rename it ata_exec_xfer().
 1.43 12-Aug-2004  thorpej Move wdc_addref() and wdc_delref() to ata.c and rename them to
ata_*.
 1.42 12-Aug-2004  thorpej Move wdc_kill_pending() to ata.c and rename it ata_kill_pending().
 1.41 12-Aug-2004  thorpej - Move wdc_xfer_pool, wdc_get_xfer(), wdc_free_xfer() to ata.c, and rename
to ata_*.
- Use a static initializer for the ata_xfer_pool.
 1.40 12-Aug-2004  thorpej Move wdc_probe_caps() to ata.c and rename it ata_probe_caps().
 1.39 12-Aug-2004  thorpej Move wdc_downgrade_mode() to ata.c and rename it to ata_downgrade_mode().
 1.38 12-Aug-2004  thorpej Move wdc_print_modes() into ata.c and rename it ata_print_modes().
 1.37 12-Aug-2004  thorpej Rename some constants:
WDC_COMPLETE -> ATACMD_COMPLETE
WDC_QUEUED -> ATACMD_QUEUED
WDC_TRY_AGAIN -> ATACMD_TRY_AGAIN
 1.36 12-Aug-2004  thorpej Rename "struct wdc_command" to "struct ata_command".
 1.35 10-Aug-2004  mycroft Hold a reference between atabus_attach() and atabusconfig() to avoid an extra
enable/disable cycle. Now my CF card attaches a bit faster.

XXX This code is split strangely between files.
 1.34 04-Aug-2004  bouyer Change wdc_kill_pending() to take a struct ata_drive_datas * as argument,
and kill only pending requests for this drive.
Implement a DRIVE_WAITDRAIN flag, which will cause the active command to
be killed once complete.
Other minor fixes.
Now it's possible to detach a ATA or ATAPI device from ioctl even when
a dd on the raw char partition is running.
 1.33 04-Aug-2004  bouyer Add
struct ata_xfer *active_xfer
to ata_queue. Now the active xfer isn't the head of the queue any more,
this makes a few things easier (this will also help for tagged queuing
support).
Remove the WDCF_ACTIVE flag, test active_xfer != NULL instead.
clean up wdc_free_xfer() and kill_xfer().
Clean up wdc_reset_channel(), and make it issue a ATAPI_SOFT_RESET if the
active command is ATAPI.
In wdc_atapi_get_params(), use AT_WAIT | AT_POLL for ATAPI_SOFT_RESET,
so that we'll use tsleep() instead of delay().
In wdc_atapi_start(), call wdc_dmawait() at the right place.
 1.32 03-Aug-2004  bouyer First cut at implementing ATABUSIODETACH. Still buggy.
 1.31 02-Aug-2004  bouyer Clean up interraction between wdc_reset_channel and the kernel thread.
Move kill_xfer() after the reset, and stop the DMA engine if needed
(this will unload the DMA maps).
 1.30 01-Aug-2004  bouyer Implement an atabus control device, and define some ATA bus control
IOCTLS. Implement ATABUSIORESET, which will reset the given ATA bus.
 1.29 27-May-2004  thorpej Clean-up of use of "precomp". It is used as "features" everywhere, so
rename the fields in wdc_command as appropriate.
 1.28 13-Apr-2004  bouyer Make wdc_channel->ch_flags volatile, and cast it to (void *) when used
in tsleep()/wakeup() to appease gcc. Otherwise, the ch_flags value may
be cached in a register in atabus_thread(), and when it sets the WDCF_TH_RUN
bit after tsleep() it may loose loose the changes made by an interrupt handler
or another thread.
Problem analysed by Jukka Andberg on tech-kern.
 1.27 03-Jan-2004  thorpej branches: 1.27.2;
More structure member namespace cleanup: thread -> ch_thread
 1.26 03-Jan-2004  thorpej More wdc_channel structure member namespace cleanup:
- channel -> ch_channel
- wdc -> ch_wdc
 1.25 03-Jan-2004  thorpej Rename "struct channel_softc" to "struct wdc_channel".
 1.24 01-Jan-2004  thorpej Rename:
- wdc_xfer to ata_xfer
- channel_queue to ata_queue
and move them to <dev/ata/atavar.h> so they can be used by non-wdc ATA
controllers. Clean up the member names of these structures while at it.
 1.23 30-Dec-2003  thorpej Move most of the atabus layer into ata.c.
 1.22 30-Dec-2003  thorpej Tweak function names in debug messages.
 1.21 14-Dec-2003  thorpej De-__P and use ANSI-style function decls.
 1.20 08-Oct-2003  bouyer Make the ATA mid-layer appears as atabus, as proposed in
http://mail-index.netbsd.org/tech-kern/2003/09/25/0006.html
This adds a device (atabus) between IDE controllers and wd or atapibus, to
have each ATA channel show up in the device tree. Later there will be atabus
devices in /dev, so that we can do IOCTL on them.
Each atabus has its own kernel thread, to handle operations that needs polling,
e.g. reset and others.

Device probing on each bus it defered to the atabus thread creation.
This allows to do the reset and basic device probes in parallel, which reduce
boot time on systems with several pciide controllers.
 1.19 23-Sep-2003  mycroft Fix more probe delay and/or failure problems:
1) Don't wait for DRQ on an IDENTIFY command -- if it's not set when we see
BSY clear, abort the command and ignore the drive. (Do this by testing
for DRQ in the read/write cases in __wdccommand_intr().)
2) Don't wait for DRQ to deassert when we finish an IDENTIFY (or any other
non-block command that reads data) -- we don't do this for block I/O, and
empirically it doesn't clear on my CF cards at all, causing a pointless 1s
delay.
3) Add comments to some of the delay()s, and add missing ones in wdcreset()
and the WDCC_RECAL in the so-called "pre-ATA" probe.
4) Slightly simplify the reset sequence -- we were doing an extra I/O.
5) Modify the register writability test to make sure that registers are not
overlapped -- this can happen in some weird cases with a missing device 1.
6) Check the error register value after the reset -- if it's not 01h or 81h,
as appropriate (see ATA spec), punt.
Tested with a number of ATA-only, ATAPI-only, mixed ATA-ATAPI, CF, and IDE
disk configurations.

Also remove the SINGLE_DRIVE nonsense again.
 1.18 27-Jan-2003  bouyer branches: 1.18.2;
Bump WDCC_IDENTIFY timeout to 3 seconds, as required by some drives.
Reported by Karl Janmar <karlj@mdstud.chalmers.se> on tech-kern.
 1.17 09-Apr-2002  bouyer branches: 1.17.4;
In wdc_exec_command(), for data commands, read the status register after
the data transfer. This is mandatory for data out commands (although none are
used for now), and not forbiddend for data in commands. Also record if we
did transfers any data.
May solve kern/16159 by making the probe more robust in face of fake identify.
 1.16 03-Dec-2001  bouyer Update my copyrigth.
 1.15 03-Dec-2001  bouyer Allow wd to attach to something else than wdc/pciide (like USB<->ATA bridges):
- move some functions from ata.c to ata_wdc.c or wdc.c.
- add callbacks to struct ata_bustype so that wd.c doesn't call directly
functions from the lower level driver.
 1.14 13-Nov-2001  lukem add RCSID
 1.13 15-May-2000  bouyer branches: 1.13.4; 1.13.6; 1.13.8;
- sync my copyrigth
- Add usefull message for obsolete error code (part of fix for kern/9856).
 1.12 25-Jan-2000  enami Don't put leading space, otherwise diskerr() prints unnecessary space
like this:
wd0h: aborted command reading fsbn 650256 of 650256-650271 (wd0 bn 839056; cn 888 tn 13 sn 17), retrying
 1.11 17-Jan-2000  bouyer Improve the downgrade logic:
- If UDMA 2 is failing try UDMA 1 first, it helps in some cases
- downgrade if we get an error in the first 4000 xfers, or if we get
4 errors in 4000 xfers if the first 4000 went without troubles.

While I'm there commit a local change I have since some time to get my CD
probed: issue a "blanck" IDENTIFY before the one used to detect slave ghosts,
with my drive the first IDENTIFY following a controller reset fails with an
aborted command ...
 1.10 26-Nov-1999  bouyer More WDCDEBUG_PRINT.
 1.9 15-Apr-1999  bouyer branches: 1.9.2; 1.9.8;
Bump timeout to 10s only for ATAPI devices, 1s is fine for IDE drives once
they are ready.
 1.8 14-Apr-1999  jonathan Complete bugfix in rev 1.3: increase read-parameter-block timeout to 10s
(as used for the hardware for which rev 1.3 was made.)
 1.7 10-Mar-1999  bouyer branches: 1.7.2;
Add a new disk flag: DRIVE_OLD for pre-ATA disks. probe routine will now set
DRIVE_OLD, DRIVE_ATA or DRIVE_ATAPI based on register signatures.
The attach routine will issue a IDENTIFY command for ATA/ATAPI disk,
to detect flase matches by the probe routine.
probe/attach should now be fully compliant with ata-4/ata-5. As a side
effect, ATAPI drives which improperly use ATA register signatures should now
be attached as ATAPI.
 1.6 21-Feb-1999  hubertf Only define WDCDEBUG if not already define (e.g. by kernel option)
 1.5 08-Feb-1999  bouyer There's no ATA draft where it is required for the drive to set DRDY | DSC when
the disk is ready to transfer data, and in ATA-5 the DSC has been obsoleted.
So only wait for DRQ to transfer data. This can be made conditional on the
ATA version if it's proven to break with some drives (worked with all the
drives I have access to).
While I'm there correct a few typos.
 1.4 18-Jan-1999  bouyer move wd.c:print_wderror() to ata.c:ata_perror().
In wdc_probe_caps() add code to guess the ATA revision supported (if
ATA4 if Ultra-DMA, ATA2 if PIO mode > 2). We can't rely on param.atap_ata_major
here, at last one Ultra-DMA drive claims to support only ATA-3.
Use the ATA version in ata_perror(), and to try a flush cache command
in a shutdown hook for IDE drives.
 1.3 16-Nov-1998  bouyer For ATAPI IDENTiFY, poll only for DRQ, some devices don't assert DRDY at this
point.
 1.2 12-Oct-1998  bouyer Merge bouyer-ide
 1.1 04-Jun-1998  bouyer branches: 1.1.2;
file ata.c was initially added on branch bouyer-ide.
 1.1.2.4 20-Sep-1998  bouyer - always call config_found() from wdcattach(), instead of printing our own
the "not configured" message ourselve. When no atapibus is configured,
use a fake ata_atapi_attach, else call wdc_atapibus_attach(). This way we
don't have to include the whole atapi support in an ata-only config.
- the dma_* function pointers take a 'int flags' as last argument, instead
of an 'int read'. The 2 valid flags are: WDC_DMA_READ (to select read/write)
and WDC_DMA_POLL, to signal interrupt-less mode (for core dumps).
- Reworked wdcprobe() so look more like the old one. A status of 0xff is
interpreted as "no drive" (freebsd does this); this this speed up
the probe for non-IDE machines (a reset timeout has been bumped to 31s,
to match the specs). The probe set ups the drive flags to either ATA or
ATAPI (depend on register signature) or OLD (if register signature don't
match, but a working drive appears to be there). Later the ATA code
will look at the OLD flag to decide wether there is no drive, or an old
(ST506) one if WDC_IDENTIFY fails.
- For known PCI chips, don't use the wdcprobe() heuristic to find wether
a channel is enabled/disabled; use the chip's specific registers for this.
- Clear the command queue when we have a polled command. This allows dumps to
work even if a command was being handled by the controller at panic time.
- Increase the number of available DMA segments by one, as user requests may
not be aligned on a page boundary.
 1.1.2.3 13-Aug-1998  bouyer - sync with HEAD
- better error detection and report for wd drives
- better wddump() support
- wdcwait() now takes a timout argument, so that we can honnor the
timeout argument of scsipi cmds.
 1.1.2.2 05-Jun-1998  bouyer Patches from Leo Weppelman for IDE on atari, with minor fixes by me
(needed the addition of a void wdcrestart __P((void*)) function).
Quoting his words:
These diffs are preliminary because it looks like the free_hw() function
does not get called. This is only a problem on the atari falcon. I have not
yet been able to track this down.
Also, KNF wdc.c.
 1.1.2.1 04-Jun-1998  bouyer Commit changes to the IDE system in a branch. This allows a better separation
between higth-level and low-level (i.e. registers read/write) and generalize
the queue for all commands. This also add supports for IDE DMA.
 1.7.2.3 07-Jul-2000  he Apply patch (requested by bouyer):
Add support for the following PCIIDE controllers:
o AMD 756
o CMD PCI0648 and PCI0649
o Hightpoint HPT366
o OPTi 82c621 (and a few of its derivatives)
o Promise Ultra/33 and Ultra/66
o Intel 82801 (ICH/ICH0)
Also fix PR#10437 (detect more ATAPI devices).
 1.7.2.2 23-Jan-2000  he Pull up revision 1.11 (requested by bouyer):
Improve the downgrade logic to work better in some cases (especially
for Acer Labs M5229 controllers with Ultra-DMA/66 disks).
 1.7.2.1 20-Apr-1999  cjs branches: 1.7.2.1.2;
Pull up to HEAD 1.9, per Manuel Bouyer <bouyer@antioche.eu.org>
 1.7.2.1.2.1 21-Jun-1999  thorpej Sync w/ -current.
 1.9.8.2 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.9.8.1 21-Dec-1999  wrstuden Initial commit of recent changes to make DEV_BSIZE go away.

Runs on i386, needs work on other arch's. Main kernel routines should be
fine, but a number of the stand programs need help.

cd, fd, ccd, wd, and sd have been updated. sd has been tested with non-512
byte block devices. vnd, raidframe, and lfs need work.

Non 2**n block support is automatic for LKM's and conditional for kernels
on "options NON_PO2_BLOCKS".
 1.9.2.1 20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
A i386 GENERIC kernel compiles without the siop, ahc and bha drivers
(will be updated later). i386 IDE/ATAPI and ncr work, as well as
sparc/esp_sbus. alpha should work as well (untested yet).
siop, ahc and bha will be updated once I've updated the branch to current
-current, as well as machine-dependant code.
 1.13.8.2 23-Jun-2002  jdolecek catch up with -current on kqueue branch
 1.13.8.1 10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.13.6.3 17-Apr-2002  nathanw Catch up to -current.
 1.13.6.2 08-Jan-2002  nathanw Catch up to -current.
 1.13.6.1 14-Nov-2001  nathanw Catch up to -current.
 1.13.4.1 23-Jan-2003  msaitoh Pull up revision 1.17 (requested by Manuel Bouyer):

read status after data commands in wdc_exec_command(). Mandatory for data out
commands (nonw are used for now), and doesn't hurt for data in. Also keep
track if we did transfer data. Makes the probe more robust and solves
kern/16159
 1.17.4.1 30-Jun-2003  grant Pull up revision 1.18 (requested by bouyer in ticket #1353):

Bump WDCC_IDENTIFY timeout to 3 seconds, as required by some drives.
Reported by Karl Janmar <karlj@mdstud.chalmers.se> on tech-kern.
 1.18.2.10 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.18.2.9 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.18.2.8 04-Feb-2005  skrll Sync with HEAD.
 1.18.2.7 29-Nov-2004  skrll Sync with HEAD.
 1.18.2.6 02-Nov-2004  skrll Sync with HEAD.
 1.18.2.5 21-Sep-2004  skrll Fix the sync with head I botched.
 1.18.2.4 18-Sep-2004  skrll Sync with HEAD.
 1.18.2.3 25-Aug-2004  skrll Sync with HEAD.
 1.18.2.2 12-Aug-2004  skrll Sync with HEAD.
 1.18.2.1 03-Aug-2004  skrll Sync with HEAD
 1.27.2.2 24-May-2005  riz Pull up revision 1.69 via patch(requested by bouyer in ticket #1543):
Close a window where (disks) interrupts could be enabled while the
ATACH_TH_RUN flag is still set. Should fix panic reported by Jukka Salmi
on netbsd-users@.
 1.27.2.1 18-Apr-2004  jmc branches: 1.27.2.1.2;
Pullup rev 1.28 (requested by bouyer in ticket #149)

Make wdc_channel->ch_flags volatile, and cast it to (void *) when used
in tsleep()/wakeup() to appease gcc. Otherwise, the ch_flags value may
be cached in a register in atabus_thread(), and when it sets the WDCF_TH_RUN
bit after tsleep() it may loose loose the changes made by an interrupt handler
or another thread.
 1.27.2.1.2.1 24-May-2005  riz Pull up revision 1.69 via patch(requested by bouyer in ticket #1543):
Close a window where (disks) interrupts could be enabled while the
ATACH_TH_RUN flag is still set. Should fix panic reported by Jukka Salmi
on netbsd-users@.
 1.63.6.2 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.63.6.1 12-Feb-2005  yamt sync with head.
 1.63.4.1 29-Apr-2005  kent sync with -current
 1.66.2.2 28-May-2005  tron Pull up revision 1.69 (requested by bouyer in ticket #340):
Close a window where (disks) interrupts could be enabled while the
ATACH_TH_RUN flag is still set. Should fix panic reported by Jukka Salmi
on netbsd-users@.
 1.66.2.1 27-May-2005  riz Pull up revision 1.68 (requested by bouyer in ticket #328):
Add a ata_queue_idle() function, which freeze a queue and tsleep() until the
controller is idle.
Change the powerhook function to call ata_queue_idle() on standby/suspend,
and ata_reset_channel() on resume (to wake up the disks and start from a
clean state).
Fix PR kern/30194 by Lubomir Sedlacik.
 1.70.2.8 24-Mar-2008  yamt sync with head.
 1.70.2.7 17-Mar-2008  yamt sync with head.
 1.70.2.6 21-Jan-2008  yamt sync with head
 1.70.2.5 27-Oct-2007  yamt sync with head.
 1.70.2.4 03-Sep-2007  yamt sync with head.
 1.70.2.3 26-Feb-2007  yamt sync with head.
 1.70.2.2 30-Dec-2006  yamt sync with head.
 1.70.2.1 21-Jun-2006  yamt sync with head.
 1.72.2.1 01-Feb-2006  yamt sync with head.
 1.73.10.1 31-Mar-2006  tron Merge 2006-03-31 NetBSD-current into the "peter-altq" branch.
 1.73.8.1 19-Apr-2006  elad sync with head.
 1.73.6.3 14-Sep-2006  yamt sync with head.
 1.73.6.2 03-Sep-2006  yamt sync with head.
 1.73.6.1 01-Apr-2006  yamt sync with head.
 1.73.4.1 22-Apr-2006  simonb Sync with head.
 1.73.2.1 09-Sep-2006  rpaulo sync with head
 1.76.4.2 10-Dec-2006  yamt sync with head.
 1.76.4.1 22-Oct-2006  yamt sync with head
 1.76.2.3 06-Feb-2007  ad Quieten noisy boot messages.
 1.76.2.2 12-Jan-2007  ad Sync with head.
 1.76.2.1 18-Nov-2006  ad Sync with head.
 1.85.2.2 24-Mar-2007  yamt sync with head.
 1.85.2.1 12-Mar-2007  rmind Sync with HEAD.
 1.86.2.8 23-Oct-2007  ad Sync with head.
 1.86.2.7 15-Jul-2007  ad Sync with head.
 1.86.2.6 01-Jul-2007  ad Adapt to callout API change.
 1.86.2.5 13-May-2007  ad - Pass the error number and residual count to biodone(), and let it handle
setting error indicators. Prepare to eliminate B_ERROR.
- Add a flag argument to brelse() to be set into the buf's flags, instead
of doing it directly. Typically used to set B_INVAL.
- Add a "struct cpu_info *" argument to kthread_create(), to be used to
create bound threads. Change "bool mpsafe" to "int flags".
- Allow exit of LWPs in the IDL state when (l != curlwp).
- More locking fixes & conversion to the new API.
 1.86.2.4 10-Apr-2007  ad Sync with head.
 1.86.2.3 10-Apr-2007  ad Nuke the deferred kthread creation stuff, as it's no longer needed.
Pointed out by thorpej@.
 1.86.2.2 09-Apr-2007  ad - Add two new arguments to kthread_create1: pri_t pri, bool mpsafe.
- Fork kthreads off proc0 as new LWPs, not new processes.
 1.86.2.1 13-Mar-2007  ad Sync with head.
 1.87.4.1 18-Mar-2007  reinoud First attempt to bring branch in sync with HEAD
 1.87.2.1 11-Jul-2007  mjf Sync with head.
 1.90.12.1 25-Oct-2007  bouyer Sync with HEAD.
 1.90.8.3 23-Mar-2008  matt sync with HEAD
 1.90.8.2 09-Jan-2008  matt sync with HEAD
 1.90.8.1 06-Nov-2007  matt sync with HEAD
 1.90.6.4 08-Dec-2007  jmcneill Rename pnp(9) -> pmf(9), as requested by many.
 1.90.6.3 06-Nov-2007  joerg Refactor PNP API:
- Make suspend/resume directly a device functionality. It consists of
three layers (class logic, device logic, bus logic), all of them being
optional. This replaces D0/D3 transitions.
- device_is_active returns true if the device was not disabled and was
not suspended (even partially), device_is_enabled returns true if the
device was enabled.
- Change pnp_global_transition into pnp_system_suspend and
pnp_system_resume. Before running any suspend/resume handlers, check
that all currently attached devices support power management and bail
out otherwise. The latter is not done for the shutdown/panic case.
- Make the former bus-specific generic network handlers a class handler.
- Make PNP message like volume up/down/toogle PNP events. Each device
can register what events they are interested in and whether the handler
should be global or not.
- Introduce device_active API for devices to mark themselve in use from
either the system or the device. Use this to implement the idle handling
for audio and input devices. This is intended to replace most ad-hoc
watchdogs as well.
- Fix somes situations in which audio resume would lose mixer settings.
- Make USB host controllers better deal with suspend in the light of
shared interrupts.
- Flush filesystem cache on suspend.
- Flush disk caches on suspend. Put ATA disks into standby on suspend as
well.
- Adopt drivers to use the new PNP API.
- Fix a critical bug in the generic cardbus layer that made D0->D3
break.
- Fix ral(4) to set if_stop.
- Convert cbb(4) to the new PNP API.
- Apply the PCI Express SCI fix on resume again.
 1.90.6.2 26-Oct-2007  joerg Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.
 1.90.6.1 03-Aug-2007  jmcneill Pull in power management changes from private branch.
 1.91.6.1 11-Dec-2007  yamt sync with head.
 1.91.4.1 26-Dec-2007  ad Sync with head.
 1.91.2.1 18-Feb-2008  mjf Sync with HEAD.
 1.92.2.3 10-Jan-2008  bouyer Sync with HEAD
 1.92.2.2 08-Jan-2008  bouyer Sync with HEAD
 1.92.2.1 02-Jan-2008  bouyer Sync with HEAD
 1.96.6.6 17-Jan-2009  mjf Sync with HEAD.
 1.96.6.5 05-Oct-2008  mjf Sync with HEAD.
 1.96.6.4 29-Jun-2008  mjf Sync with HEAD.
 1.96.6.3 06-Apr-2008  mjf - after some discussion with agc@ i agreed it would be a good idea to move
device_unregister_* to device_deregister_* to be more like the pmf(9)
functions, especially since a lot of the time the function calls are next
to each other.

- add device_register_name() support for dk(4).
 1.96.6.2 05-Apr-2008  mjf - add "file-system DEVFS" and "pseudo-device devfsctl" to conf/std seeing
as these are always needed.

- convert many, many drivers over to the New Devfs World Order. For a
list of device drivers yet to be converted see,
http://www.netbsd.org/~mjf/devfs-todo.html.

- add a new device_unregister_all(device_t) function to remove all device
names associated with a device_t, which saves us having to construct
device names when the driver is detached.

- add a DEV_AUDIO type for devices.
 1.96.6.1 03-Apr-2008  mjf Sync with HEAD.
 1.96.2.1 24-Mar-2008  keiichi sync with head.
 1.98.8.1 18-Jun-2008  simonb Sync with head.
 1.98.6.2 10-Oct-2008  skrll Sync with HEAD.
 1.98.6.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.98.4.5 11-Aug-2010  yamt sync with head.
 1.98.4.4 11-Mar-2010  yamt sync with head
 1.98.4.3 16-Sep-2009  yamt sync with head
 1.98.4.2 16-May-2009  yamt sync with head
 1.98.4.1 04-May-2009  yamt sync with head.
 1.98.2.1 17-Jun-2008  yamt sync with head.
 1.99.2.2 13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.99.2.1 19-Oct-2008  haad Sync with HEAD.
 1.101.4.1 20-Nov-2008  snj Pull up following revision(s) (requested by bouyer in ticket #67):
sys/dev/ic/wdc.c: revision 1.256
sys/dev/ata/ata.c: revision 1.102
sys/dev/scsipi/atapi_wdc.c: revision 1.109
sys/dev/ata/atavar.h: revision 1.78
sys/dev/ata/ata_wdc.c: revision 1.91
cpu_intr_p() doesn't account for software interrupts (e.g. callouts) so
we can't use it here. Rssurect ATACH_TH_RUN, backing out
src/sys/dev/ata/ata.c 1.101
src/sys/dev/ata/ata_wdc.c 1.90
src/sys/dev/ata/atavar.h 1.77
src/sys/dev/ic/wdc.c 1.255
src/sys/dev/scsipi/atapi_wdc.c 1.108
Should fix kern/39927 and kern/39725.
 1.101.2.2 28-Apr-2009  skrll Sync with HEAD.
 1.101.2.1 19-Jan-2009  skrll Sync with HEAD.
 1.102.4.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.111.2.1 30-Apr-2010  uebayasi Sync with HEAD.
 1.112.2.3 31-May-2011  rmind sync with head
 1.112.2.2 21-Apr-2011  rmind sync with head
 1.112.2.1 30-May-2010  rmind sync with head
 1.113.2.1 06-Jun-2011  jruoho Sync with HEAD.
 1.115.8.1 29-Apr-2012  mrg sync to latest -current.
 1.115.4.4 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.115.4.3 16-Jan-2013  yamt sync with (a bit old) head
 1.115.4.2 30-Oct-2012  yamt sync with head
 1.115.4.1 17-Apr-2012  yamt sync with head
 1.125.2.4 03-Dec-2017  jdolecek update from HEAD
 1.125.2.3 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.125.2.2 23-Jun-2013  tls resync from head
 1.125.2.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.127.4.2 18-May-2014  rmind sync with head
 1.127.4.1 28-Aug-2013  rmind sync with head
 1.130.2.1 10-Aug-2014  tls Rebase.
 1.132.8.39 29-Sep-2017  jdolecek introduce ATA_BSIZE and use it instead of DEV_BSIZE for get params and
recovery, where they are by spec 512 bytes and not negotiable
 1.132.8.38 27-Sep-2017  jdolecek restore the atac_claim_hw and atac_free_hw hooks, they are used on atari
 1.132.8.37 25-Sep-2017  jdolecek put recovery command on head of active xfers queue, so that drivers using
ata_queue_get_active_xfer() like mvsata(4) will find it and not random other
NCQ xfer

never return NCQ xfer from ata_queue_get_active_xfer(), if the first xfer
is NCQ simply return NULL

adjust ata_timo_xfer_check() to not check the expiring flag in C_WAITTIMO
branch, if we get there we ought to get the timeout handler aborted
 1.132.8.36 23-Sep-2017  jdolecek note to previous: the KASSERT() insisted actually there is at most one
active xfer, which is false during NCQ error recovery
 1.132.8.35 23-Sep-2017  jdolecek remove ata_queue_get_active_xfer() KASSERT() about having at least one
active xfer; it can happen we get interrupt while no longer having the cmd
active e.g. during recovery, and all callers handle getting NULL as result

fixes panic in mvsata(4) during error recovery
 1.132.8.34 20-Sep-2017  jdolecek add forgotten destroy of queue_idle cv in ata_queue_free()
 1.132.8.33 19-Sep-2017  jdolecek replace all remaining tsleep()/wakeup() calls with condition variables, or
calls to ata_delay(), as appropriate; change ata_delay() to require the
channel lock on entry, and pass the lock to kpause() for unlocking while
sleeping
 1.132.8.32 11-Sep-2017  jdolecek move debug printf where it belongs
 1.132.8.31 10-Sep-2017  jdolecek refactor code so that xfer c_start() hook is called with channel mutex held,
and hence the controller submit code no longer relies on spl

tested all the affected drivers - wdc (via piixide), ahci, mvsata, siisata,
both disk and atapi I/O
 1.132.8.30 10-Sep-2017  jdolecek fix regression in atabus_thread() when it was converted from spl to mutex -
the reset and c_start() routines expect to run on splbio; wrap the calls
insite splbio/splx() again for now, since we can't hold the mutex while calling
them

fixes problem experienced by Jonathan, where drive setup triggered
an spurious interrupt and panic due to state < READY
 1.132.8.29 15-Aug-2017  jdolecek explicitly do not try to activate any further commands when running recovery
xfer; it was kind of implied since the code would not queue another non-NCQ
command when non-NCQ command is active, but this is better for readibility
 1.132.8.28 12-Aug-2017  jdolecek use AT_LBA48 flag for the READ LOG EXT - it's required so that e.g. mvsata()
executes the command using wdccommandext(), it fails when executed using
wdccommand()
 1.132.8.27 12-Aug-2017  jdolecek take atabus_qlock while inserting into atabus_initq_head to avoid race
in attach and rescan; just cleanup, noticed this while doing the thread
locking, don't think there is realistic way to trigger this
 1.132.8.26 12-Aug-2017  jdolecek convert the atabus thread to use the channel lock and a condvar, adjust
code which sets the relevant channel flags to take the lock while doing so
 1.132.8.25 12-Aug-2017  jdolecek remove all logic around ATACH_IRQ_WAIT and channel-global ch_error/ch_status,
so that there is less hidden state shared by commands; primary intent is
to make the NCQ and non-NCQ paths more similar, and remove possibility
of incorrect handling for the NCQ commands

tested both disk and ATAPI - piixide(4) on QEMU, and siisata(4),
ahcisata(4), mvsata(4) on real hw
 1.132.8.24 01-Aug-2017  jdolecek make atastart() schedule as many commands as possible, instead of always
only one; makes it able to pick up pace again after processing non-NCQ command
 1.132.8.23 01-Aug-2017  jdolecek move the drive recovery block to drive struct, it's inherently per-drive
 1.132.8.22 29-Jul-2017  jdolecek reserve the highest slot for error recovery, and also have ata_channel
include space for the READ LOG EXT sector, so that it's not necessary
to allocate memory on the error handling path; now ata_read_log_ext_ncq()
will never fail due to resource shortage
 1.132.8.21 22-Jul-2017  jdolecek provide channel flag when executing NCQ commands, so that e.g. intr
handler can use this for handling decisions without checking xfer
 1.132.8.20 21-Jul-2017  jdolecek add checksum verification for data returned by READ LOG EXT; this is mostly
just paranoia for eventual driver/hw DMA bugs

this doesn't make difference for QEMU, as there the command actually always
just fails (it's not implemented)
 1.132.8.19 19-Jul-2017  jdolecek tighen and expand error handling, mostly for NCQ use cases:
- make retry timeout callout per xfer, i.e. retry separately
- zero whole bio struct on retry to avoid more stale state
- add a REQUEUE option, which doesn't bump retry count
- add ata_read_log_ext_ncq() for NCQ recovery
- adjust logic for activating xfers - allow next command only when
it's for same drive, several concurrent are only supported when HBA
and driver support FIS-based switching
- add new ata_timeout() which handles race between callout_stop()
and the invokation, add appropriate handling on deactivate/free paths
- stop using ch_status/ch_error in non-wdc code; later it will be dropped
completely
 1.132.8.18 27-Jun-2017  jdolecek attend error paths, more strict asserts and code consistency

- atastart() and ata_kill_pending() now KASSERT() that all xfers on queue
have same channel
- inactive xfers are killed via new reason KILL_GONE_INACTIVE, controller
code must not call any resource deactivation in that case
- c_intr() must call ata_waitdrain_xfer_check() as first thing, and must not
further touch any xfer structures on exit path; any resource cleanup
is supposed to be done in c_kill_xfer()
- c_kill_xfer() should never call atastart()
- ata_waitdrain_check() removed, replaced by ata_waitdrain_xfer_check()
- ATA_DRIVE_WAITDRAIN handling converted to use condvar
- removed unused ata_c callback
 1.132.8.17 24-Jun-2017  jdolecek change ata_delay() to tsleep for 1 hz rather than indefinitely if
provided ms is lower than 1 hz
 1.132.8.16 23-Jun-2017  jdolecek restart I/O processing after freeing xfer, i.e. now even after commands
like cache flush or standby; the command handling no longer use on-stack xfer,
hence use queue slot and compete with normal I/O for the xfers

the restart give change to all drives attached to the same channel
in round-robin fashion, for fair usage and to recover from situation
when disk is idle due to all xfers being consumed by other drives

make special concession for flush cache - ignore any new I/O requests
on the particular disk while the flush cache is waiting for xfer,
so that I/O queue won't starve the flush cache and the flush cache would
be done ASAP

tested on piixide(4), ahci(4), siisata(4)
 1.132.8.15 21-Jun-2017  jdolecek hold channel lock for ata_exec_xfer() and most of atastart(), convert
C_WAITACT handling to use condvar

add comment for the code block with C_FREE and ata_free_xfer() explaining
why it's not abstraction layer violation
 1.132.8.14 21-Jun-2017  jdolecek two last forgotten on-stack xfers replaced with using the queue xfer -
ata_get_params() and ata_set_mode()

fix wdc_atapi_get_params() to free the xfer used for soft reset before
calling ata_get_params() - it's now necessary, as ata_get_params() doesn't
invent a private xfer own any more
 1.132.8.13 21-Jun-2017  jdolecek change ata_queue_hwslot_to_xfer() and ata_queue_get_active_xfer() to take
ata_channel instead of ata_queue as parameter, and lock the channel while
traversing the queue
 1.132.8.12 20-Jun-2017  jdolecek remove ata queue downsizing - every device, attached to the same channel,
uses slots according to it's own limits

wdc code changed to expect maximum one active xfer, and not check number
of openings in the channel; this is to facilitate using wdc functions
for e.g. handling of atapi commands for drivers which support both ATAPI
and NCQ
 1.132.8.11 19-Jun-2017  jdolecek add ata_channel lock, use it to protect queue manipulation (only that for now);
add ata_channel_detach() to destroy the locks

change ata_get_xfer() so that it can wait for xfer, convert all on-stack
xfer code to use the blocking variant

fix siisata_reset_drive() to use polled reset and not try ata_activate_xfer(),
convert drive probe code also over from slot0 XXX to ata_get_xfer()

drive reset and PMP now works on siisata(4) too; changes tested also
on piixide(4), ahci(4), mvsata(4)
 1.132.8.10 17-Jun-2017  jdolecek make PMP working great again

tested with mvsata(4), my ahcisata(4) controller unfortunately doesn't
support PMP
 1.132.8.9 16-Jun-2017  jdolecek adjust reset channel and dump paths
- channel reset now always kills active transfer, even on dump path, but
now doesn't touch the queued waiting transfers; also kill_xfer hook is
always called, so that HBA can free any private xfer resources and thus
the dump request has chance to work
- kill_xfer routines now always call ata_deactivate_xfer(); added KASSERT()s
to ata_free_xfer() to expect deactivated xfer
- when called during channel reset before dump, ata_kill_active() drops
any queued waiting transfers without processing
- do not (re)queue any transfers in wddone() when dumping
- kill AT_RST_NOCMD flag

This should also hopefully fix the 'polled command has been queued' panic
as reported in:
PR kern/11811 by John Hawkinson
PR kern/47041 by Taylor R Campbell
PR kern/51979 by Martin Husemann

dump tested working with piixide(4) and ahci(4). mvsata(4) dump times out,
but otherwise tested working, will be fixed separately. siisata(4) mechanically
changed and not tested.
 1.132.8.8 24-Apr-2017  jdolecek set NCQ priority field to 'high' for BPRIO_TIMECRITICAL transfers if drive
supports it
 1.132.8.7 23-Apr-2017  jakllsch Use WDC_QUEUE_DEPTH_MASK to correctly mask the NCQ depth IDENTIFY data.
 1.132.8.6 19-Apr-2017  jdolecek add ATA FUA support
 1.132.8.5 19-Apr-2017  jdolecek adjust ata code to support more than one active command, including the
timeout handling, add support for NCQ commands

move probe for NCQ and number of tags to middle layer, negotiate mutual
support between drive and controller

implement NCQ support in ahci(4)
 1.132.8.4 15-Apr-2017  jdolecek pass also ata_command via ata_xfer, callers of ata_exec_command() is now
responsible for allocation/disposal of the structure

change code to allocate ata_xfer for commands on stack same way as previously
the ata_command were, using c_slot 0; adjust asserts so that it would allow
several xfers with same c_slot, as long as only one such transfer is active
at a time
 1.132.8.3 15-Apr-2017  jdolecek make ata_xfer's allocated as part of ata_queue and make it include ata_bio;
they are pre-allocated on attach and ata_get_xfer() now never sleep, drop the
pool

modify wd(4) to file the bio requests using the xfers and hence
make it possible to have more than one active I/O request in flight;
ata_bio callback doesn't need to allocate any memory any more,
require it to never return ATACMD_TRY_AGAIN

move lp, badsect, multi from ata_bio to ata_drive_datas, as they are per-drive,
not per transfer

drop unused drv atac_claim_hw/atac_free_hw hooks, and also drop again ata_bio
c_hwslot
 1.132.8.2 11-Apr-2017  jdolecek hide active_xfer from drivers
 1.132.8.1 10-Apr-2017  jdolecek ATA infrastructure improvements to eventually support more outstanding
commands

patch by Matt Thomas
 1.141.6.18 15-Oct-2018  jdolecek change the SATA/NCQ recovery to run in the atabus thread
 1.141.6.17 14-Oct-2018  jdolecek add comment to kthread_create() call to explain why atabus_thread is actually
running under KERNEL_LOCK()
 1.141.6.16 11-Oct-2018  jdolecek refactor shared parts of the SATA error recovery into new function
ata_recovery_resume() and use for ahcisata/siisata/mvsata, also replace
per-controller hold/unhold with generic version

move the shared recovery code into separate file ata_recovery.c
 1.141.6.15 06-Oct-2018  jdolecek actually, just make dump use the same queue skip as recovery, and remove the
no longer necessary ata_queue_reset() call from wd(4)

also for PR kern/47041
 1.141.6.14 06-Oct-2018  jdolecek remove AT_RST_EMERG, do the queue reset explicitly in wd(4)

this should explicitly fix PR kern/47041 with sync during heavy
disk activity, even thought it was actually already implicitly fixed by calling
ata_thread_run() for drive reset in previous commit already, since the
function already called ata_queue_reset()

drop now unused ch_reset_flags and drive_reset_flags
 1.141.6.13 06-Oct-2018  jdolecek remove extra newline
 1.141.6.12 06-Oct-2018  jdolecek fix ata_thread_run() for drive reset to set and check correct drive_flags
 1.141.6.11 03-Oct-2018  jdolecek change channel reset and drive reset for all ATA controllers to always
run via thread, and with channel lock held the whole time; the queue is
frozen while reset is pending

for this repurpose ata_reset_channel() into new ata_thread_run()

also adjust some device printfs to not leak xfer pointer, and avoid
aprint_* for non-autoconf messages
 1.141.6.10 24-Sep-2018  jdolecek replace KASSERT() for READ LOG EXT returning without error indication
with just log to avoid triggering panic under QEMU - it doesn't implement
the command for AHCI emulation
 1.141.6.9 22-Sep-2018  jdolecek fix use-after-free in wd(4) dump, detected by switch to the pool

change code in wd_dumpblocks() to use it's own non-pool ata_xfer,
which skips the deallocation step and thus keeps the contents when the I/O
is finished
 1.141.6.8 22-Sep-2018  jdolecek revert atabus_alloc_drives() to use KM_NOSLEEP again - it's called with
ch_lock held hence cannot actually sleep, fixing is out of scope of the branch
 1.141.6.7 22-Sep-2018  jdolecek use KM_SLEEP in atabus_alloc_drives() to avoid spurious failures; this is
not called from interrupt context
 1.141.6.6 22-Sep-2018  jdolecek separate ata_xfer slot allocation and the memory allocation, so that
there can be more queued xfers than number of supported slots by controller,
and use a pool instead of custom pre-allocation

primarily to help PR kern/52614

remove no longer needed custom wd(4) logic for flush cache

switch also wd(4) trim/suspend/setcache/wdioctlstrategy to sleep waiting
for the memory, they are all called from process context and this
avoids spurious failures
 1.141.6.5 17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.141.6.4 17-Sep-2018  jdolecek switch from TAILQ to SIMPLEQ for ata_xfer pending queue to save
space, don't need doubly linked queue
 1.141.6.3 17-Sep-2018  jdolecek move low-level protocol handlers hooks from ata_xfer to separate struct,
initialized statically

primarily to reduce ata_xfer struct size, but also improves readibility,
and enforces consistency
 1.141.6.2 01-Sep-2018  jdolecek stop assuming c_slot is immutable, use the xfer pointer rather than c_slot
in printfs()
 1.141.6.1 31-Aug-2018  jdolecek refactor ata_xfer to be just dumb structure; move all callouts/condvars out

retry callout to wd(4); reset callout and the active/cmd finish condvars
to channel queue; change code using the condvars so it works if there
are multiple waiters

simplify the async wait code for cmds, replace ata_wait_xfer()/ata_wake_xfer()
with ata_wait_cmd()

fix the callout_invoking/ack race handling code for timeouts to
actually have chance to work; change mvsata(4) to use generic timeout func

towards resolution of kern/52614
 1.141.4.4 21-Apr-2020  martin Sync with HEAD
 1.141.4.3 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.141.4.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.141.4.1 10-Jun-2019  christos Sync with HEAD
 1.141.2.2 26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.141.2.1 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.149.2.3 30-Dec-2022  martin Pull up following revision(s) (requested by tsutsui in ticket #1557):

sys/dev/ic/ahcisata_core.c: revision 1.83
sys/dev/ic/ahcisata_core.c: revision 1.102
sys/dev/ata/ata.c: revision 1.164
sys/dev/ata/ata_wdc.c: revision 1.115
sys/dev/ata/ata_recovery.c: revision 1.4
sys/dev/ic/siisata.c: revision 1.42
sys/dev/ic/wdc.c: revision 1.308
sys/dev/ic/mvsata.c: revision 1.56
sys/dev/scsipi/atapi_wdc.c: revision 1.138
sys/dev/ic/siisata.c: revision 1.49
sys/dev/ata/atavar.h: revision 1.105
sys/dev/ata/wd.c: revision 1.460
sys/dev/ata/ata.c: revision 1.155
sys/dev/ata/wd.c: revision 1.462
sys/dev/ata/atavar.h: revision 1.109
sys/dev/ata/satapmp_subr.c: revision 1.16
sys/dev/ic/wdc.c: revision 1.299
sys/dev/ic/ahcisata_core.c: revision 1.93
sys/dev/ata/ata_wdc.c: revision 1.120
sys/dev/ic/wdcvar.h: revision 1.100
sys/dev/scsipi/atapi_wdc.c: revision 1.141
sys/dev/ic/mvsata.c: revision 1.61
sys/dev/usb/umass_isdata.c (apply patch)

drop wd lock in wdstart1() before calling the ata_bio hook; when called
from ata thread context, that can still need to sleep for wdc attachments
in wdcwait()

fix use-after-free for ata xfer on bio submission found by KASAN
driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself
PR kern/55169 by Nick Hudson

Function declaration formating whitespace consistency. NFCI.

PR kern/56403
Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.
(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.
Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().
(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).
"Go ahead" by jdolecek@.
 1.149.2.2 30-Aug-2022  martin Pull up following revision(s) (requested by tsutsui in ticket #1512):

sys/dev/ata/ata.c: revision 1.163

Output missing '\n' for capability list when DMA support is not compiled in.
 1.149.2.1 15-Mar-2022  martin Pull up following revision(s) (requested by perseant in ticket #1435):

sys/dev/ata/ata.c: revision 1.167

Avoid an unaccounted extra channel freeze, if a reset is requested
more than once before the thread services the request. Closes PR#56745.
 1.154.2.2 25-Apr-2020  bouyer Sync with bouyer-xenpvh-base2 (HEAD)
 1.154.2.1 20-Apr-2020  bouyer Sync with HEAD
 1.160.4.1 02-Apr-2021  thorpej config_found_ia() -> config_found() w/ CFARG_IATTR.
 1.161.8.1 04-Aug-2021  thorpej Adapt to CFARGS().
 1.169.10.1 02-Aug-2025  perseant Sync with HEAD
 1.45 07-Aug-2021  thorpej Merge thorpej-cfargs2.
 1.44 24-Apr-2021  thorpej branches: 1.44.8;
Merge thorpej-cfargs branch:

Simplify and make extensible the config_search() / config_found() /
config_attach() interfaces: rather than having different variants for
which arguments you want pass along, just have a single call that
takes a variadic list of tag-value arguments.

Adjust all call sites:
- Simplify wherever possible; don't pass along arguments that aren't
actually needed.
- Don't be explicit about what interface attribute is attaching if
the device only has one. (More simplification.)
- Add a config_probe() function to be used in indirect configuiration
situations, making is visibly easier to see when indirect config is
in play, and allowing for future change in semantics. (As of now,
this is just a wrapper around config_match(), but that is an
implementation detail.)

Remove unnecessary or redundant interface attributes where they're not
needed.

There are currently 5 "cfargs" defined:
- CFARG_SUBMATCH (submatch function for direct config)
- CFARG_SEARCH (search function for indirect config)
- CFARG_IATTR (interface attribte)
- CFARG_LOCATORS (locators array)
- CFARG_DEVHANDLE (devhandle_t - wraps OFW, ACPI, etc. handles)

...and a sentinel value CFARG_EOL.

Add some extra sanity checking to ensure that interface attributes
aren't ambiguous.

Use CFARG_DEVHANDLE in MI FDT, OFW, and ACPI code, and macppc and shark
ports to associate those device handles with device_t instance. This
will trickle trough to more places over time (need back-end for pre-OFW
Sun OBP; any others?).
 1.43 25-Aug-2020  skrll branches: 1.43.4;
KNF
 1.42 26-Apr-2019  pgoyette Some more empty-string --> NULL conversions for module dependencies
 1.41 22-Oct-2018  jdolecek convert from malloc(9) to kmem(9)
 1.40 22-Jun-2018  pgoyette branches: 1.40.2; 1.40.4;
Call config_cfattach_attach() regardless of whether the module is
being built as part of a kernel or as a external module.

Addresses kern/53389 - thanks for the report!

XXX pullup-8
 1.39 27-Sep-2016  pgoyette branches: 1.39.6; 1.39.8; 1.39.14;
Remove some debug printf's that slipped through. Thanks, wiz!
 1.38 27-Sep-2016  pgoyette Modularize the ld driver and all of its attachments. Ensure that all
parents are capable of rescan (or otherwise provide a means of attaching
children post-initialization).
 1.37 14-Jul-2016  msaitoh branches: 1.37.2;
- Use aprint*() more in xxx_attach().
- Add missing aprint_naive("\n").
- Remove extra spaces and tabs.
- KNF.
 1.36 20-Aug-2015  christos include "ioconf.h" to get the 'void <driver>attach(int count);' prototype.
 1.35 27-Oct-2012  chs branches: 1.35.14;
split device_t/softc for all remaining drivers.
replace "struct device *" with "device_t".
use device_xname(), device_unit(), etc.
 1.34 06-Jul-2010  bsh branches: 1.34.8; 1.34.18;
fix some probles with ataraid(4) and Intel MatrixRaid.
* handle volumes largetr than 2TiB.
* support multiple arrays for Intel Matrix RAID
 1.33 12-May-2009  cegger branches: 1.33.2; 1.33.4;
KNF
 1.32 11-Sep-2008  tron branches: 1.32.8;
Add support for Intel MatrixRAID to ataraid(4). It is currently limited
to one volume.

The driver was provided by Juan RP in PR kern/39511.
 1.31 05-Sep-2008  tron Add JMicron RAID support to ataraid(4).
Patch provided by Juan RP in PR kern/39395.
 1.30 20-Aug-2008  tacha Add support for nVidia MediaShield ata raid found on HP ProLiant ML115.
Tested on HEAD and netbsd-4 branch with RAID0, RAID1, SPAN with MCP55 SATA controller.
 1.29 17-Jun-2008  reinoud branches: 1.29.2;
Mark a buffer `busy` in getnewbuf() when it came from the pool_cache since
its not on a free list.

Also change buf_init() to not automatically mark buffers `busy' since this
only makes sense for bufcache buffers.

Mark all buf_init'd buffers 'busy' on the places where they ought to be
flagged as such to not confuse the buffer cache.

Fixes PR 38923.
 1.28 06-May-2008  yamt branches: 1.28.2; 1.28.4;
ata_raid_config_block_rw: getiobuf(vp, NULL) -> getiobuf(vp, false)
 1.27 05-Apr-2008  cegger branches: 1.27.2; 1.27.4;
use aprint_*_dev and device_xname
 1.26 26-Mar-2008  matt split softc/device_t. use device_t & cfdata_t
 1.25 19-Mar-2008  cube Don't assume wd_softc embeds a struct device.
 1.24 02-Jan-2008  ad branches: 1.24.2; 1.24.6;
Merge vmlocking2 to head.
 1.23 09-Jul-2007  ad branches: 1.23.8; 1.23.14; 1.23.16; 1.23.20;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements
 1.22 27-Mar-2007  garbled Add support for VIA V-TECH ata raid. Tested on 4.0 with RAID0, RAID1 and
SPAN over a pair of sata drives on a VT8237A SATA Controller.
 1.21 09-Mar-2007  drochner branches: 1.21.2; 1.21.4; 1.21.6;
after the recent config_attach_pseudo() changes, unit numbers have the same
sementics as for real devices, so DVUNIT_ANY=-1 is illegal
 1.20 16-Nov-2006  christos branches: 1.20.2; 1.20.4;
__unused removal on arguments; approved by core.
 1.19 12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.18 04-Jan-2006  yamt branches: 1.18.18; 1.18.20;
- add simple functions to allocate/free a buffer for i/o.
- make bufpool static.
 1.17 11-Dec-2005  christos branches: 1.17.2;
merge ktrace-lwp.
 1.16 25-Aug-2005  drochner kill a number of autoconf submatch functions which follow the
standard scheme:
if (<configured> != <wildcard> && <configured> != <real>)
then fail
else
ask device match function

This is handled by config_stdsubmatch() now.
 1.15 25-Aug-2005  drochner replace the "locdesc_t" structure carrying the number of locators
explicitely by a plain integer array
the length in now known to all relevant parties, so this avoids
duplication of information, and we can allocate that thing in
drivers without hacks
 1.14 18-Jul-2005  briggs Update array of software RAID names in ata_raid.c.
Change a boundary check to ensure that we won't accidentally read and use
uninitialized memory if ATA_RAID_TYPE_MAX is updated without updating the
array.
Update comment near ATA_RAID_TYPE_MAX to note that the array in ata_raid.c
should be updated if a new ATA_RAID_TYPE_* is added.
 1.13 20-Jun-2005  briggs branches: 1.13.2;
Basic support for RAID0 and RAID1 for the Adaptec HostRAID format as found
on the Intel 6300ESB on-board RAID. Adapted from FreeBSD.
 1.12 29-May-2005  christos - remove casts [to avoid volatile]
- sprinkle const
- avoid shadowed variables
 1.11 28-Oct-2004  yamt branches: 1.11.10;
move buffer queue related stuffs from buf.h to their own header, bufq.h.
 1.10 15-Oct-2004  thorpej Change config_attach_pseudo() to take a cfdata * that contains the
necessary information to create the pseudo-device instance. Pseudo-device
device's will reference this cfdata, just as normal devices reference
their corresponding cfdata.

Welcome to 2.99.10.
 1.9 13-Sep-2004  drochner a round of autoconf cleanup:
-convert submatch() style functions (passed to config_search() or
config_found_sm()) to the locator passing variants
-pass interface attributes in some cases
-make submatch() functions look uniformly as far as possible
-avoid macros which just hide cfdata members, and reduce dependencies
on "locators.h"
 1.8 25-Jan-2004  hannken Make VOP_STRATEGY(bp) a real VOP as discussed on tech-kern.

VOP_STRATEGY(bp) is replaced by one of two new functions:

- VOP_STRATEGY(vp, bp) Call the strategy routine of vp for bp.
- DEV_STRATEGY(bp) Call the d_strategy routine of bp->b_dev for bp.

DEV_STRATEGY(bp) is used only for block-to-block device situations.
 1.7 14-Dec-2003  thorpej Garbage-collect __ATA_DISK_PRIVATE; it is no longer needed.
 1.6 29-Sep-2003  grant typo in comment
 1.5 14-Jul-2003  lukem add missing __KERNEL_RCSID()
 1.4 25-Feb-2003  thorpej branches: 1.4.2;
Add missing splbio() protecting of bufpool access. Add a clarifying
comment to <sys/buf.h> reminding everyone of the need for splbio().
 1.3 25-Feb-2003  thorpej Add a new BUF_INIT() macro which initializes b_dep and b_interlock, and
use it. This fixes a few places where either b_dep or b_interlock were
not properly initialized.
 1.2 05-Feb-2003  pk Make the buffer cache code MP-safe.
 1.1 27-Jan-2003  thorpej Experimental support for RAID volumes configured by ATA "RAID" controllers.
Such RAID controllers are actually just IDE controllers with a BIOS that
can create RAID volumes and write the configuration info to config blocks
on the disks. The BIOS can do I/O to these volumes, and the OS must
understand the config blocks and implement RAID in software in order to be
able to use these volumes.

Only SPAN (simple concatenation) and RAID0 are supported at this time,
and writing back config blocks is also not supported at this time. Currently,
only the Promise configuration scheme is supported, although supporting
the Highpoint scheme should not be too difficult.

In any case, this is sufficient to use the Promise RAID0 volume (thus
preserving the win2k AS installation) on this new Intel server I have.

Thanks to Soren Schmidt for doing the work in FreeBSD; it made this
task much easier. The config block parsing code is adapted from his
work.
 1.4.2.6 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.4.2.5 02-Nov-2004  skrll Sync with HEAD.
 1.4.2.4 19-Oct-2004  skrll Sync with HEAD
 1.4.2.3 21-Sep-2004  skrll Fix the sync with head I botched.
 1.4.2.2 18-Sep-2004  skrll Sync with HEAD.
 1.4.2.1 03-Aug-2004  skrll Sync with HEAD
 1.11.10.2 21-Jul-2005  tron Pull up revision 1.14 (requested by briggs in ticket #604):
Update array of software RAID names in ata_raid.c.
Change a boundary check to ensure that we won't accidentally read and use
uninitialized memory if ATA_RAID_TYPE_MAX is updated without updating the
array.
Update comment near ATA_RAID_TYPE_MAX to note that the array in ata_raid.c
should be updated if a new ATA_RAID_TYPE_* is added.
 1.11.10.1 03-Jul-2005  tron Pull up revision 1.13 (requested by briggs in ticket #521):
Basic support for RAID0 and RAID1 for the Adaptec HostRAID format as found
on the Intel 6300ESB on-board RAID. Adapted from FreeBSD.
 1.13.2.5 24-Mar-2008  yamt sync with head.
 1.13.2.4 21-Jan-2008  yamt sync with head
 1.13.2.3 03-Sep-2007  yamt sync with head.
 1.13.2.2 30-Dec-2006  yamt sync with head.
 1.13.2.1 21-Jun-2006  yamt sync with head.
 1.17.2.1 15-Jan-2006  yamt sync with head.
 1.18.20.2 10-Dec-2006  yamt sync with head.
 1.18.20.1 22-Oct-2006  yamt sync with head
 1.18.18.1 18-Nov-2006  ad Sync with head.
 1.20.4.2 15-Apr-2007  yamt sync with head.
 1.20.4.1 12-Mar-2007  rmind Sync with HEAD.
 1.20.2.1 31-Mar-2007  bouyer Pull up following revision(s) (requested by garbled in ticket #542):
sys/dev/ata/ata_raidreg.h: revision 1.4 via patch
sys/dev/ata/ata_raidvar.h: revision 1.5via patch
sys/dev/ata/files.ata: revision 1.15via patch
sys/dev/ata/ata_raid_via.c: revision 1.1
sys/dev/ata/ata_raid.c: revision 1.22via patch
Add support for VIA V-TECH ata raid. Tested on 4.0 with RAID0, RAID1 and
SPAN over a pair of sata drives on a VT8237A SATA Controller.
 1.21.6.1 29-Mar-2007  reinoud Pullup to -current
 1.21.4.1 11-Jul-2007  mjf Sync with head.
 1.21.2.3 24-Aug-2007  ad Sync with buffer cache locking changes. See buf.h/vfs_bio.c for details.
Some minor portions are incomplete and needs to be verified as a whole.
 1.21.2.2 10-Apr-2007  ad Sync with head.
 1.21.2.1 05-Apr-2007  ad Compile fixes.
 1.23.20.1 02-Jan-2008  bouyer Sync with HEAD
 1.23.16.1 04-Dec-2007  ad Pull the vmlocking changes into a new branch.
 1.23.14.1 18-Feb-2008  mjf Sync with HEAD.
 1.23.8.2 23-Mar-2008  matt sync with HEAD
 1.23.8.1 09-Jan-2008  matt sync with HEAD
 1.24.6.4 28-Sep-2008  mjf Sync with HEAD.
 1.24.6.3 29-Jun-2008  mjf Sync with HEAD.
 1.24.6.2 02-Jun-2008  mjf Sync with HEAD.
 1.24.6.1 03-Apr-2008  mjf Sync with HEAD.
 1.24.2.1 24-Mar-2008  keiichi sync with head.
 1.27.4.4 11-Aug-2010  yamt sync with head.
 1.27.4.3 16-May-2009  yamt sync with head
 1.27.4.2 04-May-2009  yamt sync with head.
 1.27.4.1 16-May-2008  yamt sync with head.
 1.27.2.1 18-May-2008  yamt sync with head.
 1.28.4.1 18-Jun-2008  simonb Sync with head.
 1.28.2.3 24-Sep-2008  wrstuden Merge in changes between wrstuden-revivesa-base-2 and
wrstuden-revivesa-base-3.
 1.28.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.28.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.29.2.1 19-Oct-2008  haad Sync with HEAD.
 1.32.8.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.33.4.1 05-Mar-2011  rmind sync with head
 1.33.2.1 17-Aug-2010  uebayasi Sync with HEAD.
 1.34.18.2 03-Dec-2017  jdolecek update from HEAD
 1.34.18.1 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.34.8.1 30-Oct-2012  yamt sync with head
 1.35.14.2 05-Oct-2016  skrll Sync with HEAD
 1.35.14.1 22-Sep-2015  skrll Sync with HEAD
 1.37.2.1 04-Nov-2016  pgoyette Sync with HEAD
 1.39.14.2 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.39.14.1 25-Jun-2018  pgoyette Sync with HEAD
 1.39.8.1 23-Jun-2018  martin Pull up following revision(s) (requested by pgoyette in ticket #894):

sys/dev/ata/ata_raid.c: revision 1.40

Call config_cfattach_attach() regardless of whether the module is
being built as part of a kernel or as a external module.

Addresses kern/53389 - thanks for the report!

XXX pullup-8
 1.39.6.3 17-May-2017  pgoyette It's "device_t x" not "struct device_t x"
 1.39.6.2 17-May-2017  pgoyette Typo
 1.39.6.1 17-May-2017  pgoyette At suggestion of chuq@, modify config_attach_pseudo() to return with a
reference held on the device.

Adapt callers to expect the reference to exist, and to ensure that the
reference is released.
 1.40.4.1 17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.40.2.1 10-Jun-2019  christos Sync with HEAD
 1.43.4.3 28-Mar-2021  thorpej Correct names of arguments passed to rescan function.
 1.43.4.2 22-Mar-2021  thorpej Audit CFARG_IATTR in config_found() calls, and remove it in situations
where the interface attribute is not ambiguous.
 1.43.4.1 22-Mar-2021  thorpej Mechanical conversion of config_found_sm_loc() -> config_found().
CFARG_IATTR usage needs to be audited.
 1.44.8.1 04-Aug-2021  thorpej Adapt to CFARGS().
 1.13 01-Aug-2023  mrg don't test arrays against NULL.

found by GCC 12.
 1.12 19-Mar-2022  hannken Lock vnode across VOP_OPEN.
 1.11 22-Oct-2018  jdolecek convert from malloc(9) to kmem(9)
 1.10 01-Nov-2017  mlelstv branches: 1.10.2; 1.10.4; 1.10.6;
refactor wd and ataraid drivers to use common disk subroutines.
 1.9 15-Sep-2008  tron branches: 1.9.38;
Add support for status reports via bio(4) to ataraid(4).
The code was contributed by Juan RP in PR kern/39514.
 1.8 18-Mar-2008  cube branches: 1.8.4; 1.8.6; 1.8.10;
Split device_t and softc for ATA devices, as well as wd(4). Other
cosmetic changes where appropriate.
 1.7 02-Feb-2008  mjf branches: 1.7.2; 1.7.6;
Match up some VOP_OPEN()S with VOP_CLOSE()s. This allows the root
file system to be identified properly again.
 1.6 26-Nov-2007  pooka Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern
 1.5 14-May-2006  elad branches: 1.5.32; 1.5.34; 1.5.40;
integrate kauth.
 1.4 28-Mar-2006  thorpej Use device_unit().
 1.3 25-Mar-2006  thorpej Use device_parent().
 1.2 11-Dec-2005  christos branches: 1.2.4; 1.2.6; 1.2.8; 1.2.10; 1.2.12;
merge ktrace-lwp.
 1.1 20-Jun-2005  briggs branches: 1.1.2; 1.1.4; 1.1.10;
Basic support for RAID0 and RAID1 for the Adaptec HostRAID format as found
on the Intel 6300ESB on-board RAID. Adapted from FreeBSD.
 1.1.10.2 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.1.10.1 20-Jun-2005  skrll file ata_raid_adaptec.c was added on branch ktrace-lwp on 2005-11-10 14:03:54 +0000
 1.1.4.4 24-Mar-2008  yamt sync with head.
 1.1.4.3 04-Feb-2008  yamt sync with head.
 1.1.4.2 07-Dec-2007  yamt sync with head
 1.1.4.1 21-Jun-2006  yamt sync with head.
 1.1.2.2 03-Jul-2005  tron Pull up revision 1.1 (requested by briggs in ticket #521):
Basic support for RAID0 and RAID1 for the Adaptec HostRAID format as found
on the Intel 6300ESB on-board RAID. Adapted from FreeBSD.
 1.1.2.1 20-Jun-2005  tron file ata_raid_adaptec.c was added on branch netbsd-3 on 2005-07-03 21:11:10 +0000
 1.2.12.3 24-May-2006  tron Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.
 1.2.12.2 31-Mar-2006  tron Merge 2006-03-31 NetBSD-current into the "peter-altq" branch.
 1.2.12.1 28-Mar-2006  tron Merge 2006-03-28 NetBSD-current into the "peter-altq" branch.
 1.2.10.2 06-May-2006  christos - Move kauth_cred_t declaration to <sys/types.h>
- Cleanup struct ucred; forward declarations that are unused.
- Don't include <sys/kauth.h> in any header, but include it in the c files
that need it.

Approved by core.
 1.2.10.1 19-Apr-2006  elad sync with head.
 1.2.8.2 24-May-2006  yamt sync with head.
 1.2.8.1 01-Apr-2006  yamt sync with head.
 1.2.6.2 01-Jun-2006  kardel Sync with head.
 1.2.6.1 22-Apr-2006  simonb Sync with head.
 1.2.4.1 09-Sep-2006  rpaulo sync with head
 1.5.40.2 18-Feb-2008  mjf Sync with HEAD.
 1.5.40.1 08-Dec-2007  mjf Sync with HEAD.
 1.5.34.2 23-Mar-2008  matt sync with HEAD
 1.5.34.1 09-Jan-2008  matt sync with HEAD
 1.5.32.1 27-Nov-2007  joerg Sync with HEAD. amd64 Xen support needs testing.
 1.7.6.2 28-Sep-2008  mjf Sync with HEAD.
 1.7.6.1 03-Apr-2008  mjf Sync with HEAD.
 1.7.2.1 24-Mar-2008  keiichi sync with head.
 1.8.10.1 19-Oct-2008  haad Sync with HEAD.
 1.8.6.1 24-Sep-2008  wrstuden Merge in changes between wrstuden-revivesa-base-2 and
wrstuden-revivesa-base-3.
 1.8.4.1 04-May-2009  yamt sync with head.
 1.9.38.1 03-Dec-2017  jdolecek update from HEAD
 1.10.6.1 17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.10.4.1 10-Jun-2019  christos Sync with HEAD
 1.10.2.1 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.12 01-Aug-2023  mrg don't test arrays against NULL.

found by GCC 12.
 1.11 19-Mar-2022  hannken Lock vnode across VOP_OPEN.
 1.10 04-Oct-2019  mrg use memmove() not memcpy() for overlapping regions.
 1.9 22-Oct-2018  jdolecek convert from malloc(9) to kmem(9)
 1.8 01-Nov-2017  mlelstv branches: 1.8.2; 1.8.4; 1.8.6;
refactor wd and ataraid drivers to use common disk subroutines.
 1.7 25-Mar-2014  christos kill sprintf
 1.6 06-Jul-2010  bsh branches: 1.6.8; 1.6.18; 1.6.22;
* use correct interleave factor for RAID-0 of Intel MatrixRaid.
* report stripe size in Kibytes to bioctl(8).

Unfortunately this change will break compatibility with existing
filesystem made by NetBSD on RAID-0 Intel MatrixRAID volumes.

Please read: http://mail-index.netbsd.org/current-users/2010/07/02/msg013810.html
 1.5 06-Jul-2010  bsh fix some probles with ataraid(4) and Intel MatrixRaid.
* handle volumes largetr than 2TiB.
* support multiple arrays for Intel Matrix RAID
 1.4 11-May-2009  cegger branches: 1.4.2; 1.4.4;
use device_xname()
 1.3 16-Sep-2008  tron branches: 1.3.2; 1.3.4; 1.3.6; 1.3.14; 1.3.18;
Support multiple volumes connected to Intel MatrixRAID controllers.
Code contributed by Juan RP in PR kern/39552.
 1.2 15-Sep-2008  tron Add support for status reports via bio(4) to ataraid(4).
The code was contributed by Juan RP in PR kern/39514.
 1.1 11-Sep-2008  tron Add support for Intel MatrixRAID to ataraid(4). It is currently limited
to one volume.

The driver was provided by Juan RP in PR kern/39511.
 1.3.18.4 11-Aug-2010  yamt sync with head.
 1.3.18.3 16-May-2009  yamt sync with head
 1.3.18.2 04-May-2009  yamt sync with head.
 1.3.18.1 16-Sep-2008  yamt file ata_raid_intel.c was added on branch yamt-nfs-mp on 2009-05-04 08:12:34 +0000
 1.3.14.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.3.6.2 19-Oct-2008  haad Sync with HEAD.
 1.3.6.1 16-Sep-2008  haad file ata_raid_intel.c was added on branch haad-dm on 2008-10-19 22:16:19 +0000
 1.3.4.2 28-Sep-2008  mjf Sync with HEAD.
 1.3.4.1 16-Sep-2008  mjf file ata_raid_intel.c was added on branch mjf-devfs2 on 2008-09-28 10:40:19 +0000
 1.3.2.2 24-Sep-2008  wrstuden Merge in changes between wrstuden-revivesa-base-2 and
wrstuden-revivesa-base-3.
 1.3.2.1 16-Sep-2008  wrstuden file ata_raid_intel.c was added on branch wrstuden-revivesa on 2008-09-24 16:38:51 +0000
 1.4.4.1 05-Mar-2011  rmind sync with head
 1.4.2.1 17-Aug-2010  uebayasi Sync with HEAD.
 1.6.22.1 18-May-2014  rmind sync with head
 1.6.18.2 03-Dec-2017  jdolecek update from HEAD
 1.6.18.1 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.8.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.8.6.1 17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.8.4.2 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.8.4.1 10-Jun-2019  christos Sync with HEAD
 1.8.2.1 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.9 01-Aug-2023  mrg don't test arrays against NULL.

found by GCC 12.
 1.8 19-Mar-2022  hannken Lock vnode across VOP_OPEN.
 1.7 22-Oct-2018  jdolecek convert from malloc(9) to kmem(9)
 1.6 01-Nov-2017  mlelstv branches: 1.6.2; 1.6.4; 1.6.6;
refactor wd and ataraid drivers to use common disk subroutines.
 1.5 25-Mar-2014  christos kill sprintf
 1.4 11-May-2009  cegger branches: 1.4.12; 1.4.22; 1.4.26;
use device_xname()
 1.3 15-Sep-2008  tron branches: 1.3.2; 1.3.4; 1.3.6; 1.3.14; 1.3.18;
Add support for status reports via bio(4) to ataraid(4).
The code was contributed by Juan RP in PR kern/39514.
 1.2 10-Sep-2008  tron Don't attach RAID volumes with incomplete config blocks which would lead
to panic later. Patch provided by Juan PR in private e-mail.
 1.1 05-Sep-2008  tron Add JMicron RAID support to ataraid(4).
Patch provided by Juan RP in PR kern/39395.
 1.3.18.3 16-May-2009  yamt sync with head
 1.3.18.2 04-May-2009  yamt sync with head.
 1.3.18.1 15-Sep-2008  yamt file ata_raid_jmicron.c was added on branch yamt-nfs-mp on 2009-05-04 08:12:34 +0000
 1.3.14.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.3.6.2 19-Oct-2008  haad Sync with HEAD.
 1.3.6.1 15-Sep-2008  haad file ata_raid_jmicron.c was added on branch haad-dm on 2008-10-19 22:16:19 +0000
 1.3.4.2 28-Sep-2008  mjf Sync with HEAD.
 1.3.4.1 15-Sep-2008  mjf file ata_raid_jmicron.c was added on branch mjf-devfs2 on 2008-09-28 10:40:19 +0000
 1.3.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.3.2.1 15-Sep-2008  wrstuden file ata_raid_jmicron.c was added on branch wrstuden-revivesa on 2008-09-18 04:35:02 +0000
 1.4.26.1 18-May-2014  rmind sync with head
 1.4.22.2 03-Dec-2017  jdolecek update from HEAD
 1.4.22.1 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.4.12.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.6.6.1 17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.6.4.1 10-Jun-2019  christos Sync with HEAD
 1.6.2.1 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.5 19-Mar-2022  hannken Lock vnode across VOP_OPEN.
 1.4 22-Oct-2018  jdolecek convert from malloc(9) to kmem(9)
 1.3 01-Nov-2017  mlelstv branches: 1.3.2; 1.3.4; 1.3.6;
refactor wd and ataraid drivers to use common disk subroutines.
 1.2 25-Mar-2014  christos kill sprintf
 1.1 20-Aug-2008  tacha branches: 1.1.2; 1.1.4; 1.1.6; 1.1.18; 1.1.36; 1.1.46; 1.1.52;
Add support for nVidia MediaShield ata raid found on HP ProLiant ML115.
Tested on HEAD and netbsd-4 branch with RAID0, RAID1, SPAN with MCP55 SATA controller.
 1.1.52.1 18-May-2014  rmind sync with head
 1.1.46.2 03-Dec-2017  jdolecek update from HEAD
 1.1.46.1 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.1.36.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.1.18.2 04-May-2009  yamt sync with head.
 1.1.18.1 20-Aug-2008  yamt file ata_raid_nvidia.c was added on branch yamt-nfs-mp on 2009-05-04 08:12:35 +0000
 1.1.6.2 19-Oct-2008  haad Sync with HEAD.
 1.1.6.1 20-Aug-2008  haad file ata_raid_nvidia.c was added on branch haad-dm on 2008-10-19 22:16:19 +0000
 1.1.4.2 28-Sep-2008  mjf Sync with HEAD.
 1.1.4.1 20-Aug-2008  mjf file ata_raid_nvidia.c was added on branch mjf-devfs2 on 2008-09-28 10:40:19 +0000
 1.1.2.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.1.2.1 20-Aug-2008  wrstuden file ata_raid_nvidia.c was added on branch wrstuden-revivesa on 2008-09-18 04:35:02 +0000
 1.3.6.1 17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.3.4.1 10-Jun-2019  christos Sync with HEAD
 1.3.2.1 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.14 19-Mar-2022  hannken Lock vnode across VOP_OPEN.
 1.13 22-Oct-2018  jdolecek convert from malloc(9) to kmem(9)
 1.12 01-Nov-2017  mlelstv branches: 1.12.2; 1.12.4; 1.12.6;
refactor wd and ataraid drivers to use common disk subroutines.
 1.11 18-Mar-2008  cube branches: 1.11.48;
Split device_t and softc for ATA devices, as well as wd(4). Other
cosmetic changes where appropriate.
 1.10 02-Feb-2008  mjf branches: 1.10.2; 1.10.6;
Match up some VOP_OPEN()S with VOP_CLOSE()s. This allows the root
file system to be identified properly again.
 1.9 26-Nov-2007  pooka Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern
 1.8 14-May-2006  elad branches: 1.8.32; 1.8.34; 1.8.40;
integrate kauth.
 1.7 28-Mar-2006  thorpej Use device_unit().
 1.6 11-Dec-2005  christos branches: 1.6.4; 1.6.6; 1.6.8; 1.6.10; 1.6.12;
merge ktrace-lwp.
 1.5 27-Feb-2005  perry branches: 1.5.4;
nuke trailing whitespace
 1.4 28-Oct-2004  yamt branches: 1.4.4; 1.4.6;
move buffer queue related stuffs from buf.h to their own header, bufq.h.
 1.3 14-Dec-2003  thorpej Garbage-collect __ATA_DISK_PRIVATE; it is no longer needed.
 1.2 14-Jul-2003  lukem add missing __KERNEL_RCSID()
 1.1 27-Jan-2003  thorpej branches: 1.1.2;
Experimental support for RAID volumes configured by ATA "RAID" controllers.
Such RAID controllers are actually just IDE controllers with a BIOS that
can create RAID volumes and write the configuration info to config blocks
on the disks. The BIOS can do I/O to these volumes, and the OS must
understand the config blocks and implement RAID in software in order to be
able to use these volumes.

Only SPAN (simple concatenation) and RAID0 are supported at this time,
and writing back config blocks is also not supported at this time. Currently,
only the Promise configuration scheme is supported, although supporting
the Highpoint scheme should not be too difficult.

In any case, this is sufficient to use the Promise RAID0 volume (thus
preserving the win2k AS installation) on this new Intel server I have.

Thanks to Soren Schmidt for doing the work in FreeBSD; it made this
task much easier. The config block parsing code is adapted from his
work.
 1.1.2.5 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.1.2.4 02-Nov-2004  skrll Sync with HEAD.
 1.1.2.3 21-Sep-2004  skrll Fix the sync with head I botched.
 1.1.2.2 18-Sep-2004  skrll Sync with HEAD.
 1.1.2.1 03-Aug-2004  skrll Sync with HEAD
 1.4.6.1 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.4.4.1 29-Apr-2005  kent sync with -current
 1.5.4.4 24-Mar-2008  yamt sync with head.
 1.5.4.3 04-Feb-2008  yamt sync with head.
 1.5.4.2 07-Dec-2007  yamt sync with head
 1.5.4.1 21-Jun-2006  yamt sync with head.
 1.6.12.2 24-May-2006  tron Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.
 1.6.12.1 31-Mar-2006  tron Merge 2006-03-31 NetBSD-current into the "peter-altq" branch.
 1.6.10.2 06-May-2006  christos - Move kauth_cred_t declaration to <sys/types.h>
- Cleanup struct ucred; forward declarations that are unused.
- Don't include <sys/kauth.h> in any header, but include it in the c files
that need it.

Approved by core.
 1.6.10.1 19-Apr-2006  elad sync with head.
 1.6.8.2 24-May-2006  yamt sync with head.
 1.6.8.1 01-Apr-2006  yamt sync with head.
 1.6.6.2 01-Jun-2006  kardel Sync with head.
 1.6.6.1 22-Apr-2006  simonb Sync with head.
 1.6.4.1 09-Sep-2006  rpaulo sync with head
 1.8.40.2 18-Feb-2008  mjf Sync with HEAD.
 1.8.40.1 08-Dec-2007  mjf Sync with HEAD.
 1.8.34.2 23-Mar-2008  matt sync with HEAD
 1.8.34.1 09-Jan-2008  matt sync with HEAD
 1.8.32.1 27-Nov-2007  joerg Sync with HEAD. amd64 Xen support needs testing.
 1.10.6.1 03-Apr-2008  mjf Sync with HEAD.
 1.10.2.1 24-Mar-2008  keiichi sync with head.
 1.11.48.1 03-Dec-2017  jdolecek update from HEAD
 1.12.6.1 17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.12.4.1 10-Jun-2019  christos Sync with HEAD
 1.12.2.1 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.5 16-Mar-2025  andvar ataraid(4), ld(4): ensure RAID disk status is set before resolving vnode.

Given a RAID array disk is missing (due to removal, failure, or a code bug),
its configuration is skipped. This causes ld(4) attachment at ataraid(4) to
fail because adi->adi_dev is NULL.

The fix will treat disks without known status (ADI_S_ONLINE, ADI_S_SPARE,
ADI_S_ASSIGNED) as if ata_raid_disk_vnode_find() returned NULL, more gracefully
solving above situation and avoiding the crash.

Additionally, replace vnode block device opening logic with vn_bdev_open()
(vn_subr(9)). This function provides identical functionality while also
incrementing vp->v_writecount, preventing a 'vp->v_writecount > 0` assertion
failure on vn_close().

More details are available in the tech-kern mailing list thread:
the https://mail-index.netbsd.org/tech-kern/2025/03/10/msg030179.html thread
and PR thread.

Should fix PR kern/43986
Also observed in PR kern/59130 (due to a different bug).

pullups are needed to netbsd-9, netbsd-10.
 1.4 19-Mar-2022  hannken branches: 1.4.4; 1.4.10;
Lock vnode across VOP_OPEN.
 1.3 25-Jun-2017  maxv dumb instruction
 1.2 24-Jun-2010  hannken branches: 1.2.18; 1.2.36;
Clean up vnode lock operations pass 2:

VOP_UNLOCK(vp, flags) -> VOP_UNLOCK(vp): Remove the unneeded flags argument.

Welcome to 5.99.32.

Discussed on tech-kern.
 1.1 16-Sep-2008  tron branches: 1.1.2; 1.1.4; 1.1.6; 1.1.18; 1.1.22; 1.1.24;
Support multiple volumes connected to Intel MatrixRAID controllers.
Code contributed by Juan RP in PR kern/39552.
 1.1.24.1 03-Jul-2010  rmind sync with head
 1.1.22.1 17-Aug-2010  uebayasi Sync with HEAD.
 1.1.18.3 11-Aug-2010  yamt sync with head.
 1.1.18.2 04-May-2009  yamt sync with head.
 1.1.18.1 16-Sep-2008  yamt file ata_raid_subr.c was added on branch yamt-nfs-mp on 2009-05-04 08:12:35 +0000
 1.1.6.2 19-Oct-2008  haad Sync with HEAD.
 1.1.6.1 16-Sep-2008  haad file ata_raid_subr.c was added on branch haad-dm on 2008-10-19 22:16:19 +0000
 1.1.4.2 28-Sep-2008  mjf Sync with HEAD.
 1.1.4.1 16-Sep-2008  mjf file ata_raid_subr.c was added on branch mjf-devfs2 on 2008-09-28 10:40:19 +0000
 1.1.2.2 24-Sep-2008  wrstuden Merge in changes between wrstuden-revivesa-base-2 and
wrstuden-revivesa-base-3.
 1.1.2.1 16-Sep-2008  wrstuden file ata_raid_subr.c was added on branch wrstuden-revivesa on 2008-09-24 16:38:51 +0000
 1.2.36.1 28-Aug-2017  skrll Sync with HEAD
 1.2.18.1 03-Dec-2017  jdolecek update from HEAD
 1.4.10.1 02-Aug-2025  perseant Sync with HEAD
 1.4.4.1 07-May-2025  martin Pull up following revision(s) (requested by andvar in ticket #1100):

sys/dev/ata/ld_ataraid.c: revision 1.51
sys/dev/ata/ata_raid_subr.c: revision 1.5

ataraid(4), ld(4): ensure RAID disk status is set before resolving vnode.

Given a RAID array disk is missing (due to removal, failure, or a code bug),
its configuration is skipped. This causes ld(4) attachment at ataraid(4) to
fail because adi->adi_dev is NULL.

The fix will treat disks without known status (ADI_S_ONLINE, ADI_S_SPARE,
ADI_S_ASSIGNED) as if ata_raid_disk_vnode_find() returned NULL, more gracefully
solving above situation and avoiding the crash.

Additionally, replace vnode block device opening logic with vn_bdev_open()
(vn_subr(9)). This function provides identical functionality while also
incrementing vp->v_writecount, preventing a 'vp->v_writecount > 0` assertion
failure on vn_close().

More details are available in the tech-kern mailing list thread:
the https://mail-index.netbsd.org/tech-kern/2025/03/10/msg030179.html thread
and PR thread.
Should fix PR kern/43986

Also observed in PR kern/59130 (due to a different bug).
 1.11 11-Mar-2025  andvar ataraid(4): obtain VIA V-RAID drive index from configuration blocks.

The original logic resolved disk index based on the drive channel it was
attached to. However, VX800 and some other VIA controllers have two SATA ports
per channel, which causes unintended overrides when two disks share the same
channel.

VIA V-RAID configuration blocks store the disk index as a value incrementing by
0x04 (0x00, 0x04, 0x08, 0x0C). Therefore, shift the value left by 2 obtains the
disk number.

Additionally, limit disk number to 4, which is the maximum supported by any
VIA RAID configuration.

Fixes PR kern/59130.

Tested on VX800/VT6421A/VT8251 RAID controllers.

pull-ups needed for netbsd-9, netbsd-10.
 1.10 19-Mar-2022  hannken branches: 1.10.4; 1.10.10;
Lock vnode across VOP_OPEN.
 1.9 22-Oct-2018  jdolecek branches: 1.9.4;
convert from malloc(9) to kmem(9)
 1.8 01-Nov-2017  mlelstv branches: 1.8.2; 1.8.4; 1.8.6;
refactor wd and ataraid drivers to use common disk subroutines.
 1.7 25-Mar-2014  christos kill sprintf
 1.6 19-Sep-2008  christos branches: 1.6.28; 1.6.38; 1.6.44;
PR/46593: Patrick Welche: Compute both the original and the ~VIA_T_BOOTABLE
hacked checksums and compare against both.
 1.5 18-Mar-2008  cube branches: 1.5.4; 1.5.6; 1.5.10;
Split device_t and softc for ATA devices, as well as wd(4). Other
cosmetic changes where appropriate.
 1.4 07-Feb-2008  xtraeme branches: 1.4.2; 1.4.6;
Move a debug printf into the ATA_RAID_DEBUG block.
 1.3 02-Feb-2008  mjf Match up some VOP_OPEN()S with VOP_CLOSE()s. This allows the root
file system to be identified properly again.
 1.2 26-Nov-2007  pooka Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern
 1.1 27-Mar-2007  garbled branches: 1.1.2; 1.1.4; 1.1.8; 1.1.10; 1.1.16; 1.1.18; 1.1.20; 1.1.26;
Add support for VIA V-TECH ata raid. Tested on 4.0 with RAID0, RAID1 and
SPAN over a pair of sata drives on a VT8237A SATA Controller.
 1.1.26.2 18-Feb-2008  mjf Sync with HEAD.
 1.1.26.1 08-Dec-2007  mjf Sync with HEAD.
 1.1.20.6 24-Mar-2008  yamt sync with head.
 1.1.20.5 11-Feb-2008  yamt sync with head.
 1.1.20.4 04-Feb-2008  yamt sync with head.
 1.1.20.3 07-Dec-2007  yamt sync with head
 1.1.20.2 03-Sep-2007  yamt sync with head.
 1.1.20.1 27-Mar-2007  yamt file ata_raid_via.c was added on branch yamt-lazymbuf on 2007-09-03 14:33:26 +0000
 1.1.18.2 23-Mar-2008  matt sync with HEAD
 1.1.18.1 09-Jan-2008  matt sync with HEAD
 1.1.16.1 27-Nov-2007  joerg Sync with HEAD. amd64 Xen support needs testing.
 1.1.10.2 11-Jul-2007  mjf Sync with head.
 1.1.10.1 27-Mar-2007  mjf file ata_raid_via.c was added on branch mjf-ufs-trans on 2007-07-11 20:05:17 +0000
 1.1.8.2 15-Apr-2007  yamt sync with head.
 1.1.8.1 27-Mar-2007  yamt file ata_raid_via.c was added on branch yamt-idlelwp on 2007-04-15 16:03:18 +0000
 1.1.4.2 10-Apr-2007  ad Sync with head.
 1.1.4.1 27-Mar-2007  ad file ata_raid_via.c was added on branch vmlocking on 2007-04-10 13:24:22 +0000
 1.1.2.2 31-Mar-2007  bouyer Pull up following revision(s) (requested by garbled in ticket #542):
sys/dev/ata/ata_raidreg.h: revision 1.4 via patch
sys/dev/ata/ata_raidvar.h: revision 1.5via patch
sys/dev/ata/files.ata: revision 1.15via patch
sys/dev/ata/ata_raid_via.c: revision 1.1
sys/dev/ata/ata_raid.c: revision 1.22via patch
Add support for VIA V-TECH ata raid. Tested on 4.0 with RAID0, RAID1 and
SPAN over a pair of sata drives on a VT8237A SATA Controller.
 1.1.2.1 27-Mar-2007  bouyer file ata_raid_via.c was added on branch netbsd-4 on 2007-03-31 16:38:04 +0000
 1.4.6.2 28-Sep-2008  mjf Sync with HEAD.
 1.4.6.1 03-Apr-2008  mjf Sync with HEAD.
 1.4.2.1 24-Mar-2008  keiichi sync with head.
 1.5.10.1 19-Oct-2008  haad Sync with HEAD.
 1.5.6.1 10-Oct-2008  skrll Sync with HEAD.
 1.5.4.1 04-May-2009  yamt sync with head.
 1.6.44.1 18-May-2014  rmind sync with head
 1.6.38.2 03-Dec-2017  jdolecek update from HEAD
 1.6.38.1 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.6.28.1 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.8.6.1 17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.8.4.1 10-Jun-2019  christos Sync with HEAD
 1.8.2.1 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.9.4.1 07-May-2025  martin Pull up following revision(s) (requested by andvar in ticket #1942):

sys/dev/ata/ata_raid_via.c: revision 1.11

ataraid(4): obtain VIA V-RAID drive index from configuration blocks.

The original logic resolved disk index based on the drive channel it was
attached to. However, VX800 and some other VIA controllers have two SATA ports
per channel, which causes unintended overrides when two disks share the same
channel.

VIA V-RAID configuration blocks store the disk index as a value incrementing by
0x04 (0x00, 0x04, 0x08, 0x0C). Therefore, shift the value left by 2 obtains the
disk number.

Additionally, limit disk number to 4, which is the maximum supported by any
VIA RAID configuration.

Fixes PR kern/59130.

Tested on VX800/VT6421A/VT8251 RAID controllers.
 1.10.10.1 02-Aug-2025  perseant Sync with HEAD
 1.10.4.1 07-May-2025  martin Pull up following revision(s) (requested by andvar in ticket #1099):

sys/dev/ata/ata_raid_via.c: revision 1.11

ataraid(4): obtain VIA V-RAID drive index from configuration blocks.

The original logic resolved disk index based on the drive channel it was
attached to. However, VX800 and some other VIA controllers have two SATA ports
per channel, which causes unintended overrides when two disks share the same
channel.

VIA V-RAID configuration blocks store the disk index as a value incrementing by
0x04 (0x00, 0x04, 0x08, 0x0C). Therefore, shift the value left by 2 obtains the
disk number.

Additionally, limit disk number to 4, which is the maximum supported by any
VIA RAID configuration.

Fixes PR kern/59130.

Tested on VX800/VT6421A/VT8251 RAID controllers.
 1.9 10-Sep-2014  matt g/u_int[0-9]*_t/ s/u_int/uint/g
 1.8 11-Sep-2008  tron branches: 1.8.38;
Add support for Intel MatrixRAID to ataraid(4). It is currently limited
to one volume.

The driver was provided by Juan RP in PR kern/39511.
 1.7 05-Sep-2008  tron Add JMicron RAID support to ataraid(4).
Patch provided by Juan RP in PR kern/39395.
 1.6 20-Aug-2008  tacha Add support for nVidia MediaShield ata raid found on HP ProLiant ML115.
Tested on HEAD and netbsd-4 branch with RAID0, RAID1, SPAN with MCP55 SATA controller.
 1.5 25-Dec-2007  perry branches: 1.5.6; 1.5.10; 1.5.12; 1.5.16;
Convert many of the uses of __attribute__ to equivalent
__packed, __unused and __dead macros from cdefs.h
 1.4 27-Mar-2007  garbled branches: 1.4.10; 1.4.16; 1.4.18; 1.4.22;
Add support for VIA V-TECH ata raid. Tested on 4.0 with RAID0, RAID1 and
SPAN over a pair of sata drives on a VT8237A SATA Controller.
 1.3 11-Dec-2005  christos branches: 1.3.24; 1.3.26; 1.3.30; 1.3.32; 1.3.34;
merge ktrace-lwp.
 1.2 20-Jun-2005  briggs branches: 1.2.2;
Basic support for RAID0 and RAID1 for the Adaptec HostRAID format as found
on the Intel 6300ESB on-board RAID. Adapted from FreeBSD.
 1.1 27-Jan-2003  thorpej branches: 1.1.2; 1.1.16;
Experimental support for RAID volumes configured by ATA "RAID" controllers.
Such RAID controllers are actually just IDE controllers with a BIOS that
can create RAID volumes and write the configuration info to config blocks
on the disks. The BIOS can do I/O to these volumes, and the OS must
understand the config blocks and implement RAID in software in order to be
able to use these volumes.

Only SPAN (simple concatenation) and RAID0 are supported at this time,
and writing back config blocks is also not supported at this time. Currently,
only the Promise configuration scheme is supported, although supporting
the Highpoint scheme should not be too difficult.

In any case, this is sufficient to use the Promise RAID0 volume (thus
preserving the win2k AS installation) on this new Intel server I have.

Thanks to Soren Schmidt for doing the work in FreeBSD; it made this
task much easier. The config block parsing code is adapted from his
work.
 1.1.16.1 03-Jul-2005  tron Pull up revision 1.2 (requested by briggs in ticket #521):
Basic support for RAID0 and RAID1 for the Adaptec HostRAID format as found
on the Intel 6300ESB on-board RAID. Adapted from FreeBSD.
 1.1.2.1 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.2.2.2 21-Jan-2008  yamt sync with head
 1.2.2.1 03-Sep-2007  yamt sync with head.
 1.3.34.1 29-Mar-2007  reinoud Pullup to -current
 1.3.32.1 11-Jul-2007  mjf Sync with head.
 1.3.30.1 10-Apr-2007  ad Sync with head.
 1.3.26.1 15-Apr-2007  yamt sync with head.
 1.3.24.1 31-Mar-2007  bouyer Pull up following revision(s) (requested by garbled in ticket #542):
sys/dev/ata/ata_raidreg.h: revision 1.4 via patch
sys/dev/ata/ata_raidvar.h: revision 1.5via patch
sys/dev/ata/files.ata: revision 1.15via patch
sys/dev/ata/ata_raid_via.c: revision 1.1
sys/dev/ata/ata_raid.c: revision 1.22via patch
Add support for VIA V-TECH ata raid. Tested on 4.0 with RAID0, RAID1 and
SPAN over a pair of sata drives on a VT8237A SATA Controller.
 1.4.22.1 02-Jan-2008  bouyer Sync with HEAD
 1.4.18.1 26-Dec-2007  ad Sync with head.
 1.4.16.1 18-Feb-2008  mjf Sync with HEAD.
 1.4.10.1 09-Jan-2008  matt sync with HEAD
 1.5.16.1 19-Oct-2008  haad Sync with HEAD.
 1.5.12.2 24-Sep-2008  wrstuden Merge in changes between wrstuden-revivesa-base-2 and
wrstuden-revivesa-base-3.
 1.5.12.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.5.10.1 04-May-2009  yamt sync with head.
 1.5.6.1 28-Sep-2008  mjf Sync with HEAD.
 1.8.38.1 03-Dec-2017  jdolecek update from HEAD
 1.12 06-Jul-2010  bsh fix some probles with ataraid(4) and Intel MatrixRaid.
* handle volumes largetr than 2TiB.
* support multiple arrays for Intel Matrix RAID
 1.11 12-May-2009  cegger branches: 1.11.2; 1.11.4;
struct device * -> device_t, no functional changes intended.
 1.10 16-Sep-2008  tron branches: 1.10.8;
Support multiple volumes connected to Intel MatrixRAID controllers.
Code contributed by Juan RP in PR kern/39552.
 1.9 15-Sep-2008  tron Add support for status reports via bio(4) to ataraid(4).
The code was contributed by Juan RP in PR kern/39514.
 1.8 11-Sep-2008  tron Add support for Intel MatrixRAID to ataraid(4). It is currently limited
to one volume.

The driver was provided by Juan RP in PR kern/39511.
 1.7 05-Sep-2008  tron Add JMicron RAID support to ataraid(4).
Patch provided by Juan RP in PR kern/39395.
 1.6 20-Aug-2008  tacha Add support for nVidia MediaShield ata raid found on HP ProLiant ML115.
Tested on HEAD and netbsd-4 branch with RAID0, RAID1, SPAN with MCP55 SATA controller.
 1.5 27-Mar-2007  garbled branches: 1.5.30; 1.5.34; 1.5.36; 1.5.40;
Add support for VIA V-TECH ata raid. Tested on 4.0 with RAID0, RAID1 and
SPAN over a pair of sata drives on a VT8237A SATA Controller.
 1.4 11-Dec-2005  christos branches: 1.4.24; 1.4.26; 1.4.30; 1.4.32; 1.4.34;
merge ktrace-lwp.
 1.3 18-Jul-2005  briggs Update array of software RAID names in ata_raid.c.
Change a boundary check to ensure that we won't accidentally read and use
uninitialized memory if ATA_RAID_TYPE_MAX is updated without updating the
array.
Update comment near ATA_RAID_TYPE_MAX to note that the array in ata_raid.c
should be updated if a new ATA_RAID_TYPE_* is added.
 1.2 20-Jun-2005  briggs branches: 1.2.2;
Basic support for RAID0 and RAID1 for the Adaptec HostRAID format as found
on the Intel 6300ESB on-board RAID. Adapted from FreeBSD.
 1.1 27-Jan-2003  thorpej branches: 1.1.2; 1.1.16;
Experimental support for RAID volumes configured by ATA "RAID" controllers.
Such RAID controllers are actually just IDE controllers with a BIOS that
can create RAID volumes and write the configuration info to config blocks
on the disks. The BIOS can do I/O to these volumes, and the OS must
understand the config blocks and implement RAID in software in order to be
able to use these volumes.

Only SPAN (simple concatenation) and RAID0 are supported at this time,
and writing back config blocks is also not supported at this time. Currently,
only the Promise configuration scheme is supported, although supporting
the Highpoint scheme should not be too difficult.

In any case, this is sufficient to use the Promise RAID0 volume (thus
preserving the win2k AS installation) on this new Intel server I have.

Thanks to Soren Schmidt for doing the work in FreeBSD; it made this
task much easier. The config block parsing code is adapted from his
work.
 1.1.16.2 21-Jul-2005  tron Pull up revision 1.3 (requested by briggs in ticket #604):
Update array of software RAID names in ata_raid.c.
Change a boundary check to ensure that we won't accidentally read and use
uninitialized memory if ATA_RAID_TYPE_MAX is updated without updating the
array.
Update comment near ATA_RAID_TYPE_MAX to note that the array in ata_raid.c
should be updated if a new ATA_RAID_TYPE_* is added.
 1.1.16.1 03-Jul-2005  tron Pull up revision 1.2 (requested by briggs in ticket #521):
Basic support for RAID0 and RAID1 for the Adaptec HostRAID format as found
on the Intel 6300ESB on-board RAID. Adapted from FreeBSD.
 1.1.2.1 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.2.2.2 03-Sep-2007  yamt sync with head.
 1.2.2.1 21-Jun-2006  yamt sync with head.
 1.4.34.1 29-Mar-2007  reinoud Pullup to -current
 1.4.32.1 11-Jul-2007  mjf Sync with head.
 1.4.30.1 10-Apr-2007  ad Sync with head.
 1.4.26.1 15-Apr-2007  yamt sync with head.
 1.4.24.1 31-Mar-2007  bouyer Pull up following revision(s) (requested by garbled in ticket #542):
sys/dev/ata/ata_raidreg.h: revision 1.4 via patch
sys/dev/ata/ata_raidvar.h: revision 1.5via patch
sys/dev/ata/files.ata: revision 1.15via patch
sys/dev/ata/ata_raid_via.c: revision 1.1
sys/dev/ata/ata_raid.c: revision 1.22via patch
Add support for VIA V-TECH ata raid. Tested on 4.0 with RAID0, RAID1 and
SPAN over a pair of sata drives on a VT8237A SATA Controller.
 1.5.40.1 19-Oct-2008  haad Sync with HEAD.
 1.5.36.2 24-Sep-2008  wrstuden Merge in changes between wrstuden-revivesa-base-2 and
wrstuden-revivesa-base-3.
 1.5.36.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.5.34.3 11-Aug-2010  yamt sync with head.
 1.5.34.2 16-May-2009  yamt sync with head
 1.5.34.1 04-May-2009  yamt sync with head.
 1.5.30.1 28-Sep-2008  mjf Sync with HEAD.
 1.10.8.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.11.4.1 05-Mar-2011  rmind sync with head
 1.11.2.1 17-Aug-2010  uebayasi Sync with HEAD.
 1.4 13-Apr-2020  jdolecek fix use-after-free for ata xfer on bio submission found by KASAN

driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself

PR kern/55169 by Nick Hudson
 1.3 04-Apr-2020  jdolecek branches: 1.3.2;
stop xfer timeouts during recovery, all xfers will be requeued anyway

this avoids race with the timeout routine when processing the xfers
for requeueing

should fix PR kern/54790 by Izumi Tsutsui
 1.2 22-Oct-2018  jdolecek branches: 1.2.2; 1.2.6; 1.2.8;
Merge jdolecek-ncqfixes branch

- ata_xfer's are dynamicall allocated as needed using a pool, no longer
limited to number of possible openings supported by controller; dump
and recovery paths use dedicated pre-allocated storage
- moved callouts and condvars from ata_xfer to queue or channel, so that
ata_xfer does not need special initialization
- slot allocation now done when xfer is being activated, uncoupled
from memory allocation; active slots are no longer tracked by controller
code
- channel and drive reset is done always via the atabus thread, and
now executes with channel locked the whole time
- NCQ recovery moved to shared function, and run via the thread also
- added some workarounds for buggy error recovery AHCI emulation in QEMU
and Parallels

designed to primarily fix kern/52614, but might also help with kern/47041
and kern/53183
 1.1 11-Oct-2018  jdolecek branches: 1.1.2;
file ata_recovery.c was initially added on branch jdolecek-ncqfixes.
 1.1.2.2 15-Oct-2018  jdolecek change the SATA/NCQ recovery to run in the atabus thread
 1.1.2.1 11-Oct-2018  jdolecek refactor shared parts of the SATA error recovery into new function
ata_recovery_resume() and use for ahcisata/siisata/mvsata, also replace
per-controller hold/unhold with generic version

move the shared recovery code into separate file ata_recovery.c
 1.2.8.2 30-Dec-2022  martin Pull up following revision(s) (requested by tsutsui in ticket #1557):

sys/dev/ic/ahcisata_core.c: revision 1.83
sys/dev/ic/ahcisata_core.c: revision 1.102
sys/dev/ata/ata.c: revision 1.164
sys/dev/ata/ata_wdc.c: revision 1.115
sys/dev/ata/ata_recovery.c: revision 1.4
sys/dev/ic/siisata.c: revision 1.42
sys/dev/ic/wdc.c: revision 1.308
sys/dev/ic/mvsata.c: revision 1.56
sys/dev/scsipi/atapi_wdc.c: revision 1.138
sys/dev/ic/siisata.c: revision 1.49
sys/dev/ata/atavar.h: revision 1.105
sys/dev/ata/wd.c: revision 1.460
sys/dev/ata/ata.c: revision 1.155
sys/dev/ata/wd.c: revision 1.462
sys/dev/ata/atavar.h: revision 1.109
sys/dev/ata/satapmp_subr.c: revision 1.16
sys/dev/ic/wdc.c: revision 1.299
sys/dev/ic/ahcisata_core.c: revision 1.93
sys/dev/ata/ata_wdc.c: revision 1.120
sys/dev/ic/wdcvar.h: revision 1.100
sys/dev/scsipi/atapi_wdc.c: revision 1.141
sys/dev/ic/mvsata.c: revision 1.61
sys/dev/usb/umass_isdata.c (apply patch)

drop wd lock in wdstart1() before calling the ata_bio hook; when called
from ata thread context, that can still need to sleep for wdc attachments
in wdcwait()

fix use-after-free for ata xfer on bio submission found by KASAN
driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself
PR kern/55169 by Nick Hudson

Function declaration formating whitespace consistency. NFCI.

PR kern/56403
Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.
(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.
Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().
(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).
"Go ahead" by jdolecek@.
 1.2.8.1 08-Feb-2022  martin Pull up following revision(s) (requested by jdolecek in ticket #1426):

sys/dev/ata/ata_recovery.c: revision 1.3
sys/dev/ata/ata_subr.c: revision 1.9

stop xfer timeouts during recovery, all xfers will be requeued anyway
this avoids race with the timeout routine when processing the xfers
for requeueing

should fix PR kern/54790 by Izumi Tsutsui
 1.2.6.4 21-Apr-2020  martin Sync with HEAD
 1.2.6.3 08-Apr-2020  martin Merge changes from current as of 20200406
 1.2.6.2 10-Jun-2019  christos Sync with HEAD
 1.2.6.1 22-Oct-2018  christos file ata_recovery.c was added on branch phil-wifi on 2019-06-10 22:07:05 +0000
 1.2.2.2 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.2.2.1 22-Oct-2018  pgoyette file ata_recovery.c was added on branch pgoyette-compat on 2018-11-26 01:52:30 +0000
 1.3.2.1 20-Apr-2020  bouyer Sync with HEAD
 1.13 23-Dec-2020  skrll Add missing '\n' in debug
 1.12 19-Dec-2020  jmcneill ata_timeout: restore spl in ATACH_RECOVERING path
 1.11 02-May-2020  thorpej branches: 1.11.2;
Back out changes to use a threadpool for now; it's causing trouble
for some folks on Thinkpads.
 1.10 25-Apr-2020  thorpej Rather than creating a kthread-per-channel, use a threadpool and a
threadpool-job-per-channel for the in-thread-context work that needs
to be done (which is rare).

On one of my test systems, this results in the total number of LWPs
after multi-user boot dropping from 116 to 78.
 1.9 04-Apr-2020  jdolecek branches: 1.9.2;
stop xfer timeouts during recovery, all xfers will be requeued anyway

this avoids race with the timeout routine when processing the xfers
for requeueing

should fix PR kern/54790 by Izumi Tsutsui
 1.8 07-Nov-2018  jdolecek branches: 1.8.4;
don't touch the queue structure in ata_queue_active() if atabus is already
detached from the channel

fixes panic on null pointer derefence during shutdown reported privately
by Launey Thomas
 1.7 22-Oct-2018  jdolecek Merge jdolecek-ncqfixes branch

- ata_xfer's are dynamicall allocated as needed using a pool, no longer
limited to number of possible openings supported by controller; dump
and recovery paths use dedicated pre-allocated storage
- moved callouts and condvars from ata_xfer to queue or channel, so that
ata_xfer does not need special initialization
- slot allocation now done when xfer is being activated, uncoupled
from memory allocation; active slots are no longer tracked by controller
code
- channel and drive reset is done always via the atabus thread, and
now executes with channel locked the whole time
- NCQ recovery moved to shared function, and run via the thread also
- added some workarounds for buggy error recovery AHCI emulation in QEMU
and Parallels

designed to primarily fix kern/52614, but might also help with kern/47041
and kern/53183
 1.6 10-Aug-2018  jdolecek branches: 1.6.2;
fix race in wd_lastclose() on systems with two ide disks on same
channel, which happened when one disk had pending I/O while the other
disk executed the final disk flush - need to restart bufq processing
once xfer is freed in this case

it could happen e.g. on boot when system executes fsck on different
partitions on the two drives in parallell and hence open and closes
the disk devices repeatedly

add KASSERT() for empty bufq on wd_lastclose(), and fix similar issue
also on suspend/standby path

this was introduced by the NCQ merge and not dksubr - before the merge
each drive had their own xfer, so they could not block each other

fixes PR kern/52783 by Onno van der Linden; many thanks for extensive
help with tracking this down
 1.5 06-Aug-2018  jdolecek add wddebug() which dumps some status for attached disks; indended for
debugging of PR kern/52783
 1.4 20-Oct-2017  jdolecek branches: 1.4.2; 1.4.4; 1.4.6;
move ata_queue_alloc(1) and ata_queue_free() calls to ata_channel_init()
and ata_channel_destroy() respectively, to make attachment code simpler,
and to make it easier to spot special queue manipulation like cmdide(4)

on topic of PR kern/52606
 1.3 19-Oct-2017  jdolecek more detailed debug info; also sync DEBUG_* values in wd.c with ata.c
 1.2 17-Oct-2017  jdolecek reintroduce ATACH_IRQ_WAIT flag for attachments using wdcintr(), only
process the interrupt when the flag is set - this fixes spurious interrupt
during post-reset drive setup in wdc_ata_bio_start(), and wdc_atapi_start()

while those functions set WDCTL_IDS, this seems to be ignored by certain
(maybe all) PCI-IDE controllers; usually the implicit KERNEL_LOCK() would
prevent the interrupt anyway, but not when the start routine is started
from the atabus thread, which doesn't take it

fixes 'panic: wdc_ata_bio_intr: bad state' reported on current-users
by Chavdar Ivanov
 1.1 10-Oct-2017  jdolecek split off functions used by 'wd* at umass?' into separate file, unfortunately
the previous approach with NATABUS doesn't work for kernels which include
MODULAR, but not atabus - such as macppc and evbarm
 1.4.6.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.4.6.1 10-Jun-2019  christos Sync with HEAD
 1.4.4.2 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.4.4.1 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.4.2.2 03-Dec-2017  jdolecek update from HEAD
 1.4.2.1 20-Oct-2017  jdolecek file ata_subr.c was added on branch tls-maxphys on 2017-12-03 11:36:59 +0000
 1.6.2.8 11-Oct-2018  jdolecek refactor shared parts of the SATA error recovery into new function
ata_recovery_resume() and use for ahcisata/siisata/mvsata, also replace
per-controller hold/unhold with generic version

move the shared recovery code into separate file ata_recovery.c
 1.6.2.7 06-Oct-2018  jdolecek actually, just make dump use the same queue skip as recovery, and remove the
no longer necessary ata_queue_reset() call from wd(4)

also for PR kern/47041
 1.6.2.6 22-Sep-2018  jdolecek remove explicit ata_channel_start() calls, it's no longer necessary
now that ata_xfer's are allocated via pool and not really limited;
replace by just a callout to restart the processing for rare cases
where system runs out of memory
 1.6.2.5 22-Sep-2018  jdolecek separate ata_xfer slot allocation and the memory allocation, so that
there can be more queued xfers than number of supported slots by controller,
and use a pool instead of custom pre-allocation

primarily to help PR kern/52614

remove no longer needed custom wd(4) logic for flush cache

switch also wd(4) trim/suspend/setcache/wdioctlstrategy to sleep waiting
for the memory, they are all called from process context and this
avoids spurious failures
 1.6.2.4 17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.6.2.3 17-Sep-2018  jdolecek switch from TAILQ to SIMPLEQ for ata_xfer pending queue to save
space, don't need doubly linked queue
 1.6.2.2 17-Sep-2018  jdolecek move low-level protocol handlers hooks from ata_xfer to separate struct,
initialized statically

primarily to reduce ata_xfer struct size, but also improves readibility,
and enforces consistency
 1.6.2.1 31-Aug-2018  jdolecek refactor ata_xfer to be just dumb structure; move all callouts/condvars out

retry callout to wd(4); reset callout and the active/cmd finish condvars
to channel queue; change code using the condvars so it works if there
are multiple waiters

simplify the async wait code for cmds, replace ata_wait_xfer()/ata_wake_xfer()
with ata_wait_cmd()

fix the callout_invoking/ack race handling code for timeouts to
actually have chance to work; change mvsata(4) to use generic timeout func

towards resolution of kern/52614
 1.8.4.1 08-Feb-2022  martin Pull up following revision(s) (requested by jdolecek in ticket #1426):

sys/dev/ata/ata_recovery.c: revision 1.3
sys/dev/ata/ata_subr.c: revision 1.9

stop xfer timeouts during recovery, all xfers will be requeued anyway
this avoids race with the timeout routine when processing the xfers
for requeueing

should fix PR kern/54790 by Izumi Tsutsui
 1.9.2.1 25-Apr-2020  bouyer Sync with bouyer-xenpvh-base2 (HEAD)
 1.11.2.1 03-Jan-2021  thorpej Sync w/ HEAD.
 1.120 05-Oct-2021  rin PR kern/56403

Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.

(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.

Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().

(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).

"Go ahead" by jdolecek@.
 1.119 25-Dec-2020  skrll Use designated initializers for struct ata_bustype
 1.118 21-May-2020  jdolecek branches: 1.118.2;
stop timeout handler while scheduling another part of partial I/O,
to avoid race between the timeout and I/O submission; the I/O
submission can sleep with xfer while waiting for the controller to
be ready once it gets to thread context, and timeout might cause
the xfer to be freed, leading to crashes due to use-after-free

this fixes another type of crashes with slow devices under QEMU reported
by Paul Ripke - thanks a lot with extensive debugging help
 1.117 19-May-2020  jdolecek only start the timeout machinery once the I/O is completely setup
and successful, particularly after PIO write is finished

fixes crashes in case the setup is so slow that timeout is triggered
e.g. while still waiting in wdc_wait_for_unbusy() or shortly after, without
drive actually having chance to complete the I/O, as seen in some
configuration under QEMU by Paul Ripke
 1.116 15-May-2020  jdolecek whitespace (bad indent)
 1.115 13-Apr-2020  jdolecek fix use-after-free for ata xfer on bio submission found by KASAN

driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself

PR kern/55169 by Nick Hudson
 1.114 04-Apr-2020  jdolecek branches: 1.114.2;
fix deadlock in wdcwait() when xfer timeout happens while the atabus
thread sleeps in wdcwait() - check current lwp rather than relying
on global ATACH_TH_RUN channel flag

should fix the hang part of the problem reported in
http://mail-index.netbsd.org/netbsd-users/2020/03/12/msg024249.html

thanks to Paul Ripke for providing extensive debugging info
 1.113 12-Nov-2018  jdolecek branches: 1.113.4;
hold channel lock during whole ata_dmaerr()/ata_downgrade_mode() -
according to code inspection this is safe, none of the set_modes
hooks execute anything which would be taking the lock

adresses PR kern/53714 by Andreas Gustafsson
 1.112 22-Oct-2018  jdolecek Merge jdolecek-ncqfixes branch

- ata_xfer's are dynamicall allocated as needed using a pool, no longer
limited to number of possible openings supported by controller; dump
and recovery paths use dedicated pre-allocated storage
- moved callouts and condvars from ata_xfer to queue or channel, so that
ata_xfer does not need special initialization
- slot allocation now done when xfer is being activated, uncoupled
from memory allocation; active slots are no longer tracked by controller
code
- channel and drive reset is done always via the atabus thread, and
now executes with channel locked the whole time
- NCQ recovery moved to shared function, and run via the thread also
- added some workarounds for buggy error recovery AHCI emulation in QEMU
and Parallels

designed to primarily fix kern/52614, but might also help with kern/47041
and kern/53183
 1.111 03-Sep-2018  riastradh Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)
 1.110 01-Jun-2018  macallan branches: 1.110.2; 1.110.4;
add a flag to start DMA before issuing commands - needed to work around a bug
in some SATA chips which get confused if the disk responds too fast
Mostly for K2 SATA / svwsata found in G5 Macs
adapted from OpenBSD
 1.109 17-Oct-2017  jdolecek branches: 1.109.2;
reintroduce ATACH_IRQ_WAIT flag for attachments using wdcintr(), only
process the interrupt when the flag is set - this fixes spurious interrupt
during post-reset drive setup in wdc_ata_bio_start(), and wdc_atapi_start()

while those functions set WDCTL_IDS, this seems to be ignored by certain
(maybe all) PCI-IDE controllers; usually the implicit KERNEL_LOCK() would
prevent the interrupt anyway, but not when the start routine is started
from the atabus thread, which doesn't take it

fixes 'panic: wdc_ata_bio_intr: bad state' reported on current-users
by Chavdar Ivanov
 1.108 15-Oct-2017  jdolecek revert the logic for wdc_wait_for_unbusy() in wdc_ata_bio_intr() to pre-NCQ,
wdcintr() and wdctimeout() need that; follows same change
in atapi_wdc.c rev. 1.128
 1.107 08-Oct-2017  christos Re-use the err label more so it is not unused for drivers that don't support
dma.
 1.106 07-Oct-2017  jdolecek Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.105 02-Jan-2015  christos branches: 1.105.6;
We have three sets of DTYPE_ constants in the kernel:
altq Drop Type
disklabel Disk Type
file Descriptor Type
(not to mention constants that contain the string DTYPE).
Let's make them two, by changing the disklabel one to be DisK TYPE since the
other disklabel constants seem to do that. Not many userland programs use
these constants (and the ones that they do are mostly in ifdefs). They will
be fixed shortly.
 1.104 10-Sep-2014  matt branches: 1.104.2;
g/u_int[0-9]*_t/ s/u_int/uint/g
 1.103 03-Feb-2013  jakllsch Add argument to wdccommandext() to allow the entire contents of the
device/head register to be specified. Needed for upcoming port multipler
support in mvsata(4).
 1.102 19-Nov-2012  rkujawa Introduce WDC_CAPABILITY_NO_AUXCTL flag. For lame controllers that don't
have aux control registers (driver coming soon).
 1.101 31-Jul-2012  bouyer branches: 1.101.2;
Apply back changes that were reverted on Jul 24 and Jul 26 (general ata/wdc
cleanup and SATA PMP support), now that I'm back to fix the fallouts.
 1.100 26-Jul-2012  jakllsch Revert, with intention of restoring in a less invasive way, the SATA Port
Multiplier code.

ok christos@
 1.99 24-Jul-2012  jakllsch Revert dsl@'s changes of Sun, 15 Jul 2012 10:55:35 +0000 and
Sun, 15 Jul 2012 10:56:50 +0000, excepting the kernel version bump.
First step in reverting regressions to ata(4) subsystem during the addition of
port multiplier support.
 1.98 15-Jul-2012  dsl Some namespace protection (and add greppablity).
Prefix the DRIVE_ and DRIVET_ constants from atavar.h with ATA_.
Don't use an enum for drive_type - you don't know how big it will be.
Move driver_type to avoid implicit structure padding (esp on arm).
This change is purely lexical and mechanical.

Update to 6.99.9 - this wasn't done when the SATA PMP changes
were made - I'm sure they warranted a bump.
 1.97 02-Jul-2012  bouyer Add sata Port MultiPlier (PMP) support to the ata bus layer,
as described in
http://mail-index.netbsd.org/tech-kern/2012/06/23/msg013442.html
PMP support in integrated to the atabus layer.
struct ata_channel's ch_drive[] is not dynamically allocated, and ch_ndrive
(renamed to ch_ndrives) closely reflects the size of the ch_drive[] array.
Add helper functions atabus_alloc_drives() and atabus_free_drives()
to manage ch_drive[]/ch_ndrives.
Add wdc_maxdrives to struct wdc_softc so that bus front-end can specify
how much drive they really support (master/slave or single).
ata_reset_drive() callback gains a uint32_t *sigp argument which,
when not NULL, will contain the signature of the device being reset.
While there, some cosmetic changes:
- added a drive_type enum to ata_drive_datas, and stop encoding the
probed drive type in drive_flags (we were out of drive flags anyway).
- rename DRIVE_ATAPIST to DRIVE_ATAPIDSCW to better reflect what this
really is
- remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas
- factor out the interpretation of SATA signatures in sata_interpet_sig()

propagate these changes to the ATA HBA drivers, and add support for PMP
to ahcisata(4) and siisata(4).

Thanks to:
- Protocase (http://www.protocase.com/) which provided a system
with lots of controllers, SATA PMP and drive slots
- Conservation Genomics Laboratory, Department of Biology, New Mexico State
University for hosting the above system
- Brook Milligan, who set up remote access and has been very responsive
when SATA cable move was needed
 1.96 09-Jan-2012  jakllsch Instead of assume that 'features' is 0 in wdccommandext(),
pass it as an argument, as is done for wdccommand().
 1.95 28-Aug-2011  christos branches: 1.95.2; 1.95.6;
make this compile.
 1.94 27-Aug-2011  bouyer The loongon2f+cs5526+jmicron PATA->SATA bridge cause an interresting issue:
1) because the CS5536 is not associated with a x86 CPU, interrupts are not
ack'ed as it expects so interrupts cannot configured as edge-triggered
(as is expected for a PCIIDE in compat mode)
2) the PATA->SATA bridge ignores the WDC_IDS (interrupt disable bit) so
the PATA IRQ line gets asserted when resetting or running some polled
commands. It also wrongly asserts IRQ when the (nonexistent) slave
device is selected
2) wouldn't be an issue with edge-triggered interrupt because we would
get a spurious interrupt and continue operation, a new interrupt only shows
up when the PATA IRQ line goes low and high again. But because of 1),
we get an unclearable interrupt instead, and the system loops on the
interrupt handler.

To workaround this, introduce a WDC_NO_IDS compile option which runs
all polled commands (including reset) at splbio() and without sleeps,
so that the controller's interrupt is effectively disabled and
won't be reenabled before the interrupt can be cleared.

The conditions triggering this problem are speficic enough to handle
this via a compile-time option; no need for a run-time (e.g. a
config(9), device property or callback to disable interrupts) solution.
 1.93 28-Mar-2010  snj Spell "enough" properly.
 1.92 19-Oct-2009  bouyer branches: 1.92.2; 1.92.4;
Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !
 1.91 16-Nov-2008  bouyer cpu_intr_p() doesn't account for software interrupts (e.g. callouts) so
we can't use it here. Rssurect ATACH_TH_RUN, backing out
src/sys/dev/ata/ata.c 1.101
src/sys/dev/ata/ata_wdc.c 1.90
src/sys/dev/ata/atavar.h 1.77
src/sys/dev/ic/wdc.c 1.255
src/sys/dev/scsipi/atapi_wdc.c 1.108
Should fix kern/39927 and kern/39725.
 1.90 02-Oct-2008  bouyer branches: 1.90.2; 1.90.4;
Kill ATACH_TH_RUN and use cpu_intr_p() instead.
 1.89 28-Apr-2008  martin branches: 1.89.2; 1.89.6;
Remove clause 3 and 4 from TNF licenses
 1.88 18-Mar-2008  cube branches: 1.88.2; 1.88.4;
Split device_t and softc for ATA devices, as well as wd(4). Other
cosmetic changes where appropriate.
 1.87 19-Oct-2007  ad branches: 1.87.12; 1.87.16;
machine/{bus,cpu,intr}.h -> sys/{bus,cpu,intr}.h
 1.86 08-Apr-2007  dyoung branches: 1.86.8; 1.86.10; 1.86.14;
Include opt_ata.h for ATADEBUG definition instead of #defining it
unconditionally.

Make this compile when ATADEBUG is not #defined.
 1.85 30-Sep-2006  itohy branches: 1.85.4; 1.85.8; 1.85.10;
A little effort against kernel bloat....
Exclude ATA DMA support if no ATA DMA capable drivers are compiled in.
 1.84 07-Sep-2006  itohy branches: 1.84.2; 1.84.4;
Add PIOBM (busmastering transfer using ATA PIO mode) support.
The PIOBM is used by only one driver (will be added later,
stay tuned) and intruduce an attribute "ata_piobm" so that
it will be conditionally compiled in.
The "ata_dma" (busmastering transfer using ATA DMA mode) and
"ata_udma" (busmastering transfer using ATA Ultra DMA mode)
attributes are also added for consistency, but unused for now.
 1.83 11-Dec-2005  christos branches: 1.83.4; 1.83.8;
merge ktrace-lwp.
 1.82 06-Jul-2005  thorpej branches: 1.82.2;
- When starting an ATA or ATAPI transfer, handle the case where (*dma_init)()
returns EINVAL, indicating that DMA cannot be done for this transfer.
Fall back to PIO in this case.
- Add a geodeide_dma_init() routine that checks to make sure that transfers
start on a 16 byte boundary, returning EINVAL if not. Works around a chip
bug that causes a hard system hang.

Problem reported and patch tested by Erik Fair.
 1.81 07-Jun-2005  peter Fix typos.
 1.80 29-May-2005  christos - remove casts [to avoid volatile]
- sprinkle const
- avoid shadowed variables
 1.79 27-Feb-2005  perry branches: 1.79.2;
nuke trailing whitespace
 1.78 16-Jan-2005  bouyer branches: 1.78.2;
Also call ata_dmaerr() on DMA errors (as reported by the underlying controller
driver). We want to downgrade to PIO mode if DMA isn't functionnal.
 1.77 08-Dec-2004  bouyer branches: 1.77.2;
Move ata_dmaerr() so that it's called only when there really has been an error
(ata_bio->r_error doesn't containt anything meaningfull if no error occured).
Fix kern/28255 by Izumi Tsutsui, but with a different patch.
 1.76 28-Oct-2004  yamt move buffer queue related stuffs from buf.h to their own header, bufq.h.
 1.75 21-Aug-2004  thorpej - Move to48() to atareg.h and rename it atacmd_to48().
- Add an atacmd_tosataq() that converts a 32-bit command to a Native
SATA Queued command.
 1.74 20-Aug-2004  thorpej - Add an (*ata_reset_channel)() member to ata_bustype.
- Add an ata_reset_channel() function that performs the common parts
of resetting an ATA channel, which uses the (*ata_reset_channel)()
callback to do the heavy lifting. Adjust callers to use ata_reset_channel()
instead of wdc_reset_channel().

This removes the last wdc-specific code from ata.c!
 1.73 20-Aug-2004  thorpej Move most of wdc_softc into a new atac_softc structure that contains
info common to all types of ATA controllers.
 1.72 14-Aug-2004  thorpej - Split the register handles out of struct wdc_channel into a separate
wdc_regs structure, and array of which (indexed per channel) is pointed
to by struct wdc_softc.
- Move the resulting wdc_channel structure to atavar.h and rename it to
ata_channel. Rename the corresponding flags.
- Add a "ch_ndrive" member to struct ata_channel, which indicates the
maximum number of drives that can be present on the channel. For now,
this is always 2. Add an ATA_MAXDRIVES constant that places an upper
limit on this value, also currently 2.
 1.71 13-Aug-2004  thorpej WDCDEBUG -> ATADEBUG.
 1.70 13-Aug-2004  thorpej Don't bother with bits that tell of the presence of optional callbacks;
just check the function pointers for NULL.
 1.69 13-Aug-2004  thorpej Move wdcstart() to ata.c and rename it to atastart().
 1.68 13-Aug-2004  thorpej Move wdc_exec_xfer() to ata.c and rename it ata_exec_xfer().
 1.67 12-Aug-2004  thorpej Move wdc_addref() and wdc_delref() to ata.c and rename them to
ata_*.
 1.66 12-Aug-2004  thorpej Move wdc_kill_pending() to ata.c and rename it ata_kill_pending().
 1.65 12-Aug-2004  thorpej - Move wdc_xfer_pool, wdc_get_xfer(), wdc_free_xfer() to ata.c, and rename
to ata_*.
- Use a static initializer for the ata_xfer_pool.
 1.64 12-Aug-2004  thorpej Rename some constants:
WDC_COMPLETE -> ATACMD_COMPLETE
WDC_QUEUED -> ATACMD_QUEUED
WDC_TRY_AGAIN -> ATACMD_TRY_AGAIN
 1.63 11-Aug-2004  mycroft Make datain_pio and dataout_pio function pointers in wdc_softc, which can
be overridden by the backend if desired. Add experimental code to wdc_pcmcia
to use this in memory-mapped mode, disabled by default.
 1.62 11-Aug-2004  mycroft Add two helper functions -- wdc_datain_pio() and wdc_dataout_pio() -- which
encapsulate the logic for the various methods of transferring data. Use these
throughout.
 1.61 04-Aug-2004  bouyer Change wdc_kill_pending() to take a struct ata_drive_datas * as argument,
and kill only pending requests for this drive.
Implement a DRIVE_WAITDRAIN flag, which will cause the active command to
be killed once complete.
Other minor fixes.
Now it's possible to detach a ATA or ATAPI device from ioctl even when
a dd on the raw char partition is running.
 1.60 04-Aug-2004  bouyer Add
struct ata_xfer *active_xfer
to ata_queue. Now the active xfer isn't the head of the queue any more,
this makes a few things easier (this will also help for tagged queuing
support).
Remove the WDCF_ACTIVE flag, test active_xfer != NULL instead.
clean up wdc_free_xfer() and kill_xfer().
Clean up wdc_reset_channel(), and make it issue a ATAPI_SOFT_RESET if the
active command is ATAPI.
In wdc_atapi_get_params(), use AT_WAIT | AT_POLL for ATAPI_SOFT_RESET,
so that we'll use tsleep() instead of delay().
In wdc_atapi_start(), call wdc_dmawait() at the right place.
 1.59 01-Aug-2004  bouyer Implement an atabus control device, and define some ATA bus control
IOCTLS. Implement ATABUSIORESET, which will reset the given ATA bus.
 1.58 31-Jul-2004  bouyer Implement asynchronous channel reset.
Use this to reset the channel before doing a dump, instead of the hack in
wdc_exec_xfer() based on C_POLL. This hack was causing problems on
controllers with a shared queue, because we now can have C_POLL set during
concurent channels probes (problem found and analysed on sparc64 by
Martin Husemann).
This should even make core dumps marginally more reliable on ATA drives.
 1.57 22-Jun-2004  mycroft Update copyright.
 1.56 22-Jun-2004  mycroft Do not force ATA_SINGLE on devices that don't support READ/WRITE MULTI. We
just have to take an interrupt for each sector.
Tested with one laptop disk (which normally runs in DMA mode and was forced
to single-sector transfers) and 3 CF cards. Increases the performance of
the CF cards substantially (760KB/s->1240KB/s in one case, 410KB/s->750KB/s
in the other two cases).
 1.55 01-Jun-2004  mycroft Only downgrade modes due to an actual CRC error. Downgrading on other errors
is anti-social -- especially given that there's no way to upgrade again short
of rebooting.

Also, downgrade UDMA modes more slowly. It's entirely possible that they're
using an 80-wire cable, but it's just too long for the higher modes, or there
is minor crosstalk.
 1.54 08-May-2004  bouyer Add a delay(10) after re-enabling interrupts in the control register.
Some controllers/drives (e.g. SataLink 3114 with WD Raptor) require
it. Should fix kern/23808 by Chris Gilbert, patch suplied by Chris Gilbert
on tech-kern, extended to all places enabling interrupts by me.
 1.53 02-Mar-2004  fvdl branches: 1.53.2;
During a control operation (like a reset), wait 10us after selecting
the drive. This fixes (or rather, works around) a timing problem
with WD Raptor drives attached to a Sil3114 SATA controller.

Should fix PR 23808, it fixes the same problem for me.
 1.52 03-Jan-2004  thorpej More structure member namespace cleanup: thread -> ch_thread
 1.51 03-Jan-2004  thorpej More wdc_channel structure member namespace cleanup:
- channel -> ch_channel
- wdc -> ch_wdc
 1.50 03-Jan-2004  thorpej Rename "struct channel_softc" to "struct wdc_channel".
 1.49 01-Jan-2004  thorpej Prepend "wdc_" to wait_for_drq, wait_for_unbusy, and wait_for_ready.
 1.48 01-Jan-2004  thorpej Rename:
- wdc_xfer to ata_xfer
- channel_queue to ata_queue
and move them to <dev/ata/atavar.h> so they can be used by non-wdc ATA
controllers. Clean up the member names of these structures while at it.
 1.47 14-Dec-2003  thorpej No need to include wdvar.h anymore.
 1.46 14-Dec-2003  thorpej Instead of calling wddone() directly, set a (*drv_done)() callback in
the ata_drive_datas structure and invoke that.
 1.45 14-Dec-2003  thorpej Liberally apply static.
 1.44 14-Dec-2003  thorpej De-__P and use ANSI-style function decls.
 1.43 27-Nov-2003  fvdl There are some cards that map the ATA control and IDE DMA registers
in a different fashion. Individually, they have the same functionality,
but their layout is different. An example of such a chipset is
the Promise 203xx.

To be able to deal with this, transform the cmd and dma bus_space handles
into an array of handles, each seperately created with bus_space_subregion.
The code generated by using the extra indirection shouldn't change much,
since the extra indirection is negated by having the offset calculation
already done in bus_space_subregion. E.g.

bus_space_write_4(tag, handle, offset, value)

becomes

bus_space_write_4(tag, handles[offset], 0, value)

Reviewed by Manuel Bouyer. Tested on wdc_isa, wdc_pcmcia, viaide, piixide (i386)
and on cmdide (sparc64).
 1.42 29-Oct-2003  bouyer freese->freeze, as pointed out by Frederick Bruck.
 1.41 08-Oct-2003  bouyer Make the ATA mid-layer appears as atabus, as proposed in
http://mail-index.netbsd.org/tech-kern/2003/09/25/0006.html
This adds a device (atabus) between IDE controllers and wd or atapibus, to
have each ATA channel show up in the device tree. Later there will be atabus
devices in /dev, so that we can do IOCTL on them.
Each atabus has its own kernel thread, to handle operations that needs polling,
e.g. reset and others.

Device probing on each bus it defered to the atabus thread creation.
This allows to do the reset and basic device probes in parallel, which reduce
boot time on systems with several pciide controllers.
 1.40 05-Oct-2003  bouyer Remove references to University of California from my copyright notices.
 1.39 28-Apr-2003  nakayama branches: 1.39.2;
Make DMA mode works on Promise Ultra66/100 with 48-bit LBA drives.
Ok'ed by bouyer in tech-kern@netbsd.org.
 1.38 27-Sep-2002  provos remove trailing \n in panic(). approved perry.
 1.37 26-Jul-2002  wiz enouth -> enough.
 1.36 23-Apr-2002  bouyer branches: 1.36.2; 1.36.4;
More copyright fixes, pointed out by Thomas. Thanks !
 1.35 14-Jan-2002  christos add the dma command ext translations. pointed out by manuel.
 1.34 13-Jan-2002  christos Add LBA48 support based on patches from Keisuke YOSHIDA. KNF a bit while
I am there.
 1.33 03-Dec-2001  bouyer Update my copyrigth.
 1.32 03-Dec-2001  bouyer Allow wd to attach to something else than wdc/pciide (like USB<->ATA bridges):
- move some functions from ata.c to ata_wdc.c or wdc.c.
- add callbacks to struct ata_bustype so that wd.c doesn't call directly
functions from the lower level driver.
 1.31 13-Nov-2001  lukem add RCSID
 1.30 13-Jun-2001  bjh21 branches: 1.30.2;
Add explicit support for IDE and SCSI adaptors which don't support interrupts.
On such adaptors, all transfers are done in polling mode.

OK'ed by Manuel on tech-kern.
 1.29 18-Apr-2001  jdolecek fix typo in comment - s/splio()/splbio()/
 1.28 12-Jun-2000  bouyer branches: 1.28.4;
Add a callback (*irqack), for controllers that need special action to ack
the interrupt once it has been ack'd on the drive.
 1.27 15-May-2000  bouyer branches: 1.27.2;
Sync my copyrigth notice
 1.26 01-Apr-2000  bouyer - DMA code cleanup: pciide_dma_finish() doesn't stop/unload the current DMA op
if an IRQ was not detected, unless the force flag was given. Use this to
detect if the IRQ was for us (closer to shared IRQ for controllers which
don't have their own IRQ handler in pciide.c) and to poll for DMA xfer.
Also makes the timeout recovery code simpler.
- ATAPI cleanup: don't call controller-specific functions from atapiconf.c
(wdc_*), so that it's possible to attach an atapibus to something else
than a wdc/pciide (Hi Lennart :).
Overload struct scsi_adapter with struct atapi_adapter, defined
as struct scsi_adapter + atapi-specific callbacks. scsipi_link still points
to an scsi_adapter, atapi code casts it to atapi_adapter if needed.
Move atapi_softc to atapiconf.h so that it can be used by the underlying
controller code (e.g. atapi_wdc.c).
Add an atapi-specific callback *atapi_probedev(), which probe a drive
in a controller-specific way, allocate the sc_link and fills in the
ataparams if needed. It then calls atapi_probedev() (from atapiconf.c)
to do the generic initialisations and attach the device.
- While I'm there merge and centralise the state definitions in atavar.h.
It should now be possible to use a common ata/atapi routine to set the
drive's modes (will do later).
 1.25 25-Mar-2000  bouyer Remove unneeded assignement to nblks
 1.24 23-Mar-2000  thorpej New callout mechanism with two major improvements over the old
timeout()/untimeout() API:
- Clients supply callout handle storage, thus eliminating problems of
resource allocation.
- Insertion and removal of callouts is constant time, important as
this facility is used quite a lot in the kernel.

The old timeout()/untimeout() API has been removed from the kernel.
 1.23 17-Jan-2000  bouyer Improve the downgrade logic:
- If UDMA 2 is failing try UDMA 1 first, it helps in some cases
- downgrade if we get an error in the first 4000 xfers, or if we get
4 errors in 4000 xfers if the first 4000 went without troubles.

While I'm there commit a local change I have since some time to get my CD
probed: issue a "blanck" IDENTIFY before the one used to detect slave ghosts,
with my drive the first IDENTIFY following a controller reset fails with an
aborted command ...
 1.22 20-Oct-1999  enami Cancel active transfers on aic/wdc detach.
Also makes LS-120 drive works for me again.
 1.21 09-Aug-1999  bouyer branches: 1.21.2; 1.21.4; 1.21.6;
Don't use C_NEEDDONE, it's deprecated.
 1.20 06-Aug-1999  bouyer - Add some debug printf (WDCPROBE) in _wdcreset_wait(), I've needed these
2 times in the past
- Set up timeout per xfer instead of per interrupt. This helps with
PIO transfer (we would call timeout()/untimout() several times for a
transfer).
- If we missed an interrupt for a PIO transfer, reset and restart it
immedialy, don't try to recover and continue. If we missed an interrupt we
may have lost a read/write cycle on the IDE bus. If this happens 1) we
corrupt data and 2) we enter an interrupt loop at the end of the xfer, as
the drive has some more data to read/write, but the host thinks the xfer is
done.
This last change fix the (or at last some of the) 'lookup after lost interrupt'
some peoples have been experiencing.
 1.19 01-Apr-1999  bouyer branches: 1.19.2;
- change the interrupt routines to take a 3rd arguments, set to 1 if we
are called from the interrupt or timeout handler, 0 otherwise.
- use this to know if we can busy-wait for wait_for_unbusy or wait_for_ready
This fixes a bug where CDs withot the DRQ_INTR capability would not busy-wait
for the CMDOUT phase.
While I'm there change 2 delay() to DELAY() for consistency, and
garbage-collect some old code from wdcintr() which has been ifdef'd out
for some time now.
 1.18 25-Mar-1999  bouyer Avoid busy-waiting when possible if we are not polling (real IRQ or timeout
callback). Shared PCI IRQ should now work (but still untested).
 1.17 17-Mar-1999  bouyer Revert to a 10s timeout, 1s is too low for drives in sleep mode.
 1.16 07-Mar-1999  bouyer Keep track of CRC errors in Ultra-DMA mode. If we noticed a CRC error and we
need to downgrade, downgrade to PIO, as it has been shown if we got CRC errors
in Ultra-DMA mode, we will have silent data corruption in multiword DMA mode
(isn't IDE wonderfull ? :).
Set timeout to 1s for "normal" ata I/O, to minimise the effects of missed
interrupts.
 1.15 21-Feb-1999  hubertf Only define WDCDEBUG if not already define (e.g. by kernel option)
 1.14 08-Feb-1999  bouyer There's no ATA draft where it is required for the drive to set DRDY | DSC when
the disk is ready to transfer data, and in ATA-5 the DSC has been obsoleted.
So only wait for DRQ to transfer data. This can be made conditional on the
ATA version if it's proven to break with some drives (worked with all the
drives I have access to).
While I'm there correct a few typos.
 1.13 29-Jan-1999  bouyer Cosmetic changes, from Soren S. Jorvan.
 1.12 09-Jan-1999  thorpej Make this compile again after void * arithmetic changes.
 1.11 08-Jan-1999  augustss Avoid arithmetic on `void *' since that's not ANSI C.
 1.10 16-Dec-1998  bouyer Keep track of DMA errors, and downgrade the transert mode (UDMA ->DMA,
DMA->PIO) in case of 2 consecutive errors. Don't downgrade if the
PIO/DMA/UDMA modes were forced by a config flag.
 1.9 21-Nov-1998  drochner in wdc_softc: access the per-channel data via a pointer array instead of
an array of fixed-sized channel_softc elements. This way IDE controllers
which more than 1 channel (pciide) can extend the channel data easily
for private needs.
To avoid the double dereference at runtime, change the argument of
wdcstart() to the channel data pointer instead of the array index.
 1.8 20-Nov-1998  thorpej Implement controller reference counting on ATA devices.
 1.7 11-Nov-1998  bouyer - clearify the boot messages (features supported vs features used). Thanks to
Havard Eidnes for his complains about this :)
- fix some typo in comments
- hoppefully better detection of drives reporting bogus PIO modes.
 1.6 13-Oct-1998  bouyer Add a new drive flag, DRIVE_MODE, set when the drive properly reported its
PIO/DMA modes. Don't try a SET_FEATURE if it didn't report its mode.
 1.5 13-Oct-1998  bouyer In debug message, always print controller:channel:drive
 1.4 13-Oct-1998  bouyer Properly separate DEBUF_XFERS and DEBUG_FUNCS: DEBUF_XFERS is for data
transfers (may be called often), where DEBUG_FUNCS is for setup functions
(not used for normal contitions).
 1.3 13-Oct-1998  thorpej Default to "no debugging messages" (still patchable).
 1.2 12-Oct-1998  bouyer Merge bouyer-ide
 1.1 04-Jun-1998  bouyer branches: 1.1.2;
file ata_wdc.c was initially added on branch bouyer-ide.
 1.1.2.15 05-Oct-1998  bouyer Rename WDC_CAPABILITY_PIO to WDC_CAPABILITY_MODE, as it's used to tell wether
the controller's driver can tell which timing mode it uses. Check this
before setting DMA modes too.
This allow the generic DMA code to work again.
 1.1.2.14 04-Oct-1998  bouyer Remove an unneeded delay().
 1.1.2.13 04-Oct-1998  bouyer New flag for wdc_xfer: C_DMA, set when a cmd will use DMA. This reduce some
if() in ata_wdc.c and atapi_wdc.c from 3 tests to one.
 1.1.2.12 02-Oct-1998  bouyer Cleanup a few error message, remove some dead code. Re-add a reset at end of
attach, unless we have WDC_NO_EXTRA_RESETS.
Add support for data32iot/data32ioh from -current: each controller pass
WDC_CAPABILITY_DATA32 if they can do 32bit, and WDC_CAPABILITY_DATA16 if they
can do 16bit. For controller that support both, the usual autodetect
mechanism is still used.
 1.1.2.11 20-Sep-1998  bouyer Init up xfer->drive in wdc_ata_bio(), or bad things will happen for drive 1.
No need to init xfer->channel, it's done in wdc_exec_xfer().
 1.1.2.10 20-Sep-1998  bouyer - always call config_found() from wdcattach(), instead of printing our own
the "not configured" message ourselve. When no atapibus is configured,
use a fake ata_atapi_attach, else call wdc_atapibus_attach(). This way we
don't have to include the whole atapi support in an ata-only config.
- the dma_* function pointers take a 'int flags' as last argument, instead
of an 'int read'. The 2 valid flags are: WDC_DMA_READ (to select read/write)
and WDC_DMA_POLL, to signal interrupt-less mode (for core dumps).
- Reworked wdcprobe() so look more like the old one. A status of 0xff is
interpreted as "no drive" (freebsd does this); this this speed up
the probe for non-IDE machines (a reset timeout has been bumped to 31s,
to match the specs). The probe set ups the drive flags to either ATA or
ATAPI (depend on register signature) or OLD (if register signature don't
match, but a working drive appears to be there). Later the ATA code
will look at the OLD flag to decide wether there is no drive, or an old
(ST506) one if WDC_IDENTIFY fails.
- For known PCI chips, don't use the wdcprobe() heuristic to find wether
a channel is enabled/disabled; use the chip's specific registers for this.
- Clear the command queue when we have a polled command. This allows dumps to
work even if a command was being handled by the controller at panic time.
- Increase the number of available DMA segments by one, as user requests may
not be aligned on a page boundary.
 1.1.2.9 21-Aug-1998  bouyer Sync with -current
 1.1.2.8 13-Aug-1998  bouyer - sync with HEAD
- better error detection and report for wd drives
- better wddump() support
- wdcwait() now takes a timout argument, so that we can honnor the
timeout argument of scsipi cmds.
 1.1.2.7 23-Jun-1998  leo Add necessary cruft to handle swapped byte-lanes on the data-register. It
basically boils down to the fact that there are 2 new options:
WDC_CAPABILITY_{ATA,ATAPI}_NOSTREAM with which you are able to control
the usage bus_space_read_multi_X() vs. bus_space_read_multi_stream_X().
 1.1.2.6 09-Jun-1998  bouyer If an error occured while setting up the drive, reset the drive's state
to 0.
 1.1.2.5 09-Jun-1998  bouyer - sync with trunck
- don't try to set the drive's PIO mode if we don't know how to set it on the
controller. This seems to confuse some 'intelligent' ISA controllers.
 1.1.2.4 05-Jun-1998  bouyer _DMA -> _UDMA, + cosmetic changes.
Ultra DMA still doesn't work.
 1.1.2.3 05-Jun-1998  bouyer - call wdc_ata_bio_done() only once.
- better error recovery: if an error occured, retry the transfer in
single sector PIO mode.
 1.1.2.2 05-Jun-1998  bouyer A few bux fixes (untested):
- If the PIO mode of the controller is unknown, put the drive in the
hightest mode it supports. A drive in a faster mode than the controller
should work.
- Create DMA maps even for unknown controllers in pciide.
 1.1.2.1 04-Jun-1998  bouyer Commit changes to the IDE system in a branch. This allows a better separation
between higth-level and low-level (i.e. registers read/write) and generalize
the queue for all commands. This also add supports for IDE DMA.
 1.19.2.3 07-Jul-2000  he Apply patch (requested by bouyer):
Add support for the following PCIIDE controllers:
o AMD 756
o CMD PCI0648 and PCI0649
o Hightpoint HPT366
o OPTi 82c621 (and a few of its derivatives)
o Promise Ultra/33 and Ultra/66
o Intel 82801 (ICH/ICH0)
Also fix PR#10437 (detect more ATAPI devices).
 1.19.2.2 23-Jan-2000  he Pull up revision 1.23 (requested by bouyer):
Improve the downgrade logic to work better in some cases (especially
for Acer Labs M5229 controllers with Ultra-DMA/66 disks).
 1.19.2.1 25-Aug-1999  he Pull up revision 1.19->1.20:
Add some debug printfs, change timeout handling to be more efficient
and less error-prone, and fix the "lockup after lost interrupt"
problem some users have been reporting. (bouyer)
 1.21.6.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.21.4.1 15-Nov-1999  fvdl Sync with -current
 1.21.2.3 21-Apr-2001  bouyer Sync with HEAD
 1.21.2.2 20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
A i386 GENERIC kernel compiles without the siop, ahc and bha drivers
(will be updated later). i386 IDE/ATAPI and ncr work, as well as
sparc/esp_sbus. alpha should work as well (untested yet).
siop, ahc and bha will be updated once I've updated the branch to current
-current, as well as machine-dependant code.
 1.21.2.1 20-Oct-1999  thorpej Sync w/ trunk.
 1.27.2.1 22-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.28.4.7 18-Oct-2002  nathanw Catch up to -current.
 1.28.4.6 01-Aug-2002  nathanw Catch up to -current.
 1.28.4.5 20-Jun-2002  nathanw Catch up to -current.
 1.28.4.4 28-Feb-2002  nathanw Catch up to -current.
 1.28.4.3 08-Jan-2002  nathanw Catch up to -current.
 1.28.4.2 14-Nov-2001  nathanw Catch up to -current.
 1.28.4.1 21-Jun-2001  nathanw Catch up to -current.
 1.30.2.5 10-Oct-2002  jdolecek sync kqueue with -current; this includes merge of gehenna-devsw branch,
merge of i386 MP branch, and part of autoconf rototil work
 1.30.2.4 06-Sep-2002  jdolecek sync kqueue branch with HEAD
 1.30.2.3 23-Jun-2002  jdolecek catch up with -current on kqueue branch
 1.30.2.2 11-Feb-2002  jdolecek Sync w/ -current.
 1.30.2.1 10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.36.4.1 28-Apr-2003  tron Pull up revision 1.39 (requested by nakayama in ticket #1275):
Make DMA mode works on Promise Ultra66/100 with 48-bit LBA drives.
Ok'ed by bouyer in tech-kern@netbsd.org.
 1.36.2.1 29-Aug-2002  gehenna catch up with -current.
 1.39.2.10 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.39.2.9 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.39.2.8 17-Jan-2005  skrll Sync with HEAD.
 1.39.2.7 18-Dec-2004  skrll Sync with HEAD.
 1.39.2.6 02-Nov-2004  skrll Sync with HEAD.
 1.39.2.5 21-Sep-2004  skrll Fix the sync with head I botched.
 1.39.2.4 18-Sep-2004  skrll Sync with HEAD.
 1.39.2.3 25-Aug-2004  skrll Sync with HEAD.
 1.39.2.2 12-Aug-2004  skrll Sync with HEAD.
 1.39.2.1 03-Aug-2004  skrll Sync with HEAD
 1.53.2.3 11-Aug-2004  jmc branches: 1.53.2.3.2;
Pullup rev 1.58 (requested by bouyer in ticket #733)

Implement asynchronous channel reset.
Use this to reset the channel before doing a dump, instead of the hack in
wdc_exec_xfer() based on C_POLL. This hack was causing problems on
controllers with a shared queue, because we now can have C_POLL set during
concurent channels probes
 1.53.2.2 02-Jul-2004  he Pull up revisions 1.56-1.57 (requested by mycroft in ticket #572):
Several fixes mostly related to USB:
o Don't force ATA_SINGLE on devices which don't support
READ/WRITE MULTI. Increases performance with several
CF cards substantially.
o Update copyright.
 1.53.2.1 29-May-2004  tron Pull up revision 1.54 (requested by bouyer in ticket #396):
Add a delay(10) after re-enabling interrupts in the control register.
Some controllers/drives (e.g. SataLink 3114 with WD Raptor) require
it. Should fix kern/23808 by Chris Gilbert, patch suplied by Chris Gilbert
on tech-kern, extended to all places enabling interrupts by me.
 1.53.2.3.2.4 18-Jul-2005  riz Pull up revision 1.82 (requested by thorpej in ticket #2072):
- When starting an ATA or ATAPI transfer, handle the case where
(*dma_init)()
returns EINVAL, indicating that DMA cannot be done for this
transfer.
Fall back to PIO in this case.
- Add a geodeide_dma_init() routine that checks to make sure that
transfers
start on a 16 byte boundary, returning EINVAL if not. Works
around a chip
bug that causes a hard system hang.
Problem reported and patch tested by Erik Fair.
 1.53.2.3.2.3 16-Mar-2005  tron Pull up revision 1.78 (requested by bouyer in ticket #1325):
Also call ata_dmaerr() on DMA errors (as reported by the underlying controller
driver). We want to downgrade to PIO mode if DMA isn't functionnal.
 1.53.2.3.2.2 16-Mar-2005  tron Pull up revision 1.77 (requested by bouyer in ticket #1324):
Move ata_dmaerr() so that it's called only when there really has been an error
(ata_bio->r_error doesn't containt anything meaningfull if no error occured).
Fix kern/28255 by Izumi Tsutsui, but with a different patch.
 1.53.2.3.2.1 16-Mar-2005  jmc Pullup rev 1.55 (requested by mycroft in ticket #1018)

Only downgrade modes due to an actual CRC error. Downgrading on other errors
is anti-social -- especially given that there's no way to upgrade again short
of rebooting.

Also, downgrade UDMA modes more slowly. It's entirely possible that they're
using an 80-wire cable, but it's just too long for the higher modes, or there
is minor crosstalk.
 1.77.2.1 29-Apr-2005  kent sync with -current
 1.78.2.1 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.79.2.1 06-Jul-2005  tron Pull up revision 1.82 (requested by thorpej in ticket #553):
- When starting an ATA or ATAPI transfer, handle the case where (*dma_init)()
returns EINVAL, indicating that DMA cannot be done for this transfer.
Fall back to PIO in this case.
- Add a geodeide_dma_init() routine that checks to make sure that transfers
start on a 16 byte boundary, returning EINVAL if not. Works around a chip
bug that causes a hard system hang.
Problem reported and patch tested by Erik Fair.
 1.82.2.4 24-Mar-2008  yamt sync with head.
 1.82.2.3 27-Oct-2007  yamt sync with head.
 1.82.2.2 03-Sep-2007  yamt sync with head.
 1.82.2.1 30-Dec-2006  yamt sync with head.
 1.83.8.1 14-Sep-2006  yamt sync with head.
 1.83.4.1 09-Sep-2006  rpaulo sync with head
 1.84.4.1 22-Oct-2006  yamt sync with head
 1.84.2.1 18-Nov-2006  ad Sync with head.
 1.85.10.1 11-Jul-2007  mjf Sync with head.
 1.85.8.2 23-Oct-2007  ad Sync with head.
 1.85.8.1 10-Apr-2007  ad Sync with head.
 1.85.4.1 15-Apr-2007  yamt sync with head.
 1.86.14.1 25-Oct-2007  bouyer Sync with HEAD.
 1.86.10.2 23-Mar-2008  matt sync with HEAD
 1.86.10.1 06-Nov-2007  matt sync with HEAD
 1.86.8.1 26-Oct-2007  joerg Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.
 1.87.16.4 17-Jan-2009  mjf Sync with HEAD.
 1.87.16.3 05-Oct-2008  mjf Sync with HEAD.
 1.87.16.2 02-Jun-2008  mjf Sync with HEAD.
 1.87.16.1 03-Apr-2008  mjf Sync with HEAD.
 1.87.12.1 24-Mar-2008  keiichi sync with head.
 1.88.4.3 11-Aug-2010  yamt sync with head.
 1.88.4.2 11-Mar-2010  yamt sync with head
 1.88.4.1 16-May-2008  yamt sync with head.
 1.88.2.1 18-May-2008  yamt sync with head.
 1.89.6.2 13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.89.6.1 19-Oct-2008  haad Sync with HEAD.
 1.89.2.1 10-Oct-2008  skrll Sync with HEAD.
 1.90.4.1 20-Nov-2008  snj Pull up following revision(s) (requested by bouyer in ticket #67):
sys/dev/ic/wdc.c: revision 1.256
sys/dev/ata/ata.c: revision 1.102
sys/dev/scsipi/atapi_wdc.c: revision 1.109
sys/dev/ata/atavar.h: revision 1.78
sys/dev/ata/ata_wdc.c: revision 1.91
cpu_intr_p() doesn't account for software interrupts (e.g. callouts) so
we can't use it here. Rssurect ATACH_TH_RUN, backing out
src/sys/dev/ata/ata.c 1.101
src/sys/dev/ata/ata_wdc.c 1.90
src/sys/dev/ata/atavar.h 1.77
src/sys/dev/ic/wdc.c 1.255
src/sys/dev/scsipi/atapi_wdc.c 1.108
Should fix kern/39927 and kern/39725.
 1.90.2.1 19-Jan-2009  skrll Sync with HEAD.
 1.92.4.1 30-May-2010  rmind sync with head
 1.92.2.1 30-Apr-2010  uebayasi Sync with HEAD.
 1.95.6.1 18-Feb-2012  mrg merge to -current.
 1.95.2.4 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.95.2.3 16-Jan-2013  yamt sync with (a bit old) head
 1.95.2.2 30-Oct-2012  yamt sync with head
 1.95.2.1 17-Apr-2012  yamt sync with head
 1.101.2.2 03-Dec-2017  jdolecek update from HEAD
 1.101.2.1 25-Feb-2013  tls resync with head
 1.104.2.1 06-Apr-2015  skrll Sync with HEAD
 1.105.6.12 30-Sep-2017  jdolecek must drop channel lock before calling ata_dmaerr() to avoid 'locking against
myself' in case of consecutive errors from ata_reset_channel() called via
ata_downgrade_mode()
 1.105.6.11 26-Sep-2017  jdolecek make compile without NATA_DMA
 1.105.6.10 21-Sep-2017  jdolecek add missing ata_channel_unlock() in bailout path of wdc_ata_bio_intr()
 1.105.6.9 10-Sep-2017  jdolecek refactor code so that xfer c_start() hook is called with channel mutex held,
and hence the controller submit code no longer relies on spl

tested all the affected drivers - wdc (via piixide), ahci, mvsata, siisata,
both disk and atapi I/O
 1.105.6.8 12-Aug-2017  jdolecek convert the atabus thread to use the channel lock and a condvar, adjust
code which sets the relevant channel flags to take the lock while doing so
 1.105.6.7 12-Aug-2017  jdolecek remove all logic around ATACH_IRQ_WAIT and channel-global ch_error/ch_status,
so that there is less hidden state shared by commands; primary intent is
to make the NCQ and non-NCQ paths more similar, and remove possibility
of incorrect handling for the NCQ commands

tested both disk and ATAPI - piixide(4) on QEMU, and siisata(4),
ahcisata(4), mvsata(4) on real hw
 1.105.6.6 27-Jun-2017  jdolecek attend error paths, more strict asserts and code consistency

- atastart() and ata_kill_pending() now KASSERT() that all xfers on queue
have same channel
- inactive xfers are killed via new reason KILL_GONE_INACTIVE, controller
code must not call any resource deactivation in that case
- c_intr() must call ata_waitdrain_xfer_check() as first thing, and must not
further touch any xfer structures on exit path; any resource cleanup
is supposed to be done in c_kill_xfer()
- c_kill_xfer() should never call atastart()
- ata_waitdrain_check() removed, replaced by ata_waitdrain_xfer_check()
- ATA_DRIVE_WAITDRAIN handling converted to use condvar
- removed unused ata_c callback
 1.105.6.5 20-Jun-2017  jdolecek remove ata queue downsizing - every device, attached to the same channel,
uses slots according to it's own limits

wdc code changed to expect maximum one active xfer, and not check number
of openings in the channel; this is to facilitate using wdc functions
for e.g. handling of atapi commands for drivers which support both ATAPI
and NCQ
 1.105.6.4 16-Jun-2017  jdolecek adjust reset channel and dump paths
- channel reset now always kills active transfer, even on dump path, but
now doesn't touch the queued waiting transfers; also kill_xfer hook is
always called, so that HBA can free any private xfer resources and thus
the dump request has chance to work
- kill_xfer routines now always call ata_deactivate_xfer(); added KASSERT()s
to ata_free_xfer() to expect deactivated xfer
- when called during channel reset before dump, ata_kill_active() drops
any queued waiting transfers without processing
- do not (re)queue any transfers in wddone() when dumping
- kill AT_RST_NOCMD flag

This should also hopefully fix the 'polled command has been queued' panic
as reported in:
PR kern/11811 by John Hawkinson
PR kern/47041 by Taylor R Campbell
PR kern/51979 by Martin Husemann

dump tested working with piixide(4) and ahci(4). mvsata(4) dump times out,
but otherwise tested working, will be fixed separately. siisata(4) mechanically
changed and not tested.
 1.105.6.3 19-Apr-2017  jdolecek adjust ata code to support more than one active command, including the
timeout handling, add support for NCQ commands

move probe for NCQ and number of tags to middle layer, negotiate mutual
support between drive and controller

implement NCQ support in ahci(4)
 1.105.6.2 15-Apr-2017  jdolecek make ata_xfer's allocated as part of ata_queue and make it include ata_bio;
they are pre-allocated on attach and ata_get_xfer() now never sleep, drop the
pool

modify wd(4) to file the bio requests using the xfers and hence
make it possible to have more than one active I/O request in flight;
ata_bio callback doesn't need to allocate any memory any more,
require it to never return ATACMD_TRY_AGAIN

move lp, badsect, multi from ata_bio to ata_drive_datas, as they are per-drive,
not per transfer

drop unused drv atac_claim_hw/atac_free_hw hooks, and also drop again ata_bio
c_hwslot
 1.105.6.1 10-Apr-2017  jdolecek ATA infrastructure improvements to eventually support more outstanding
commands

patch by Matt Thomas
 1.109.2.3 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.109.2.2 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.109.2.1 25-Jun-2018  pgoyette Sync with HEAD
 1.110.4.4 15-Oct-2018  jdolecek change the SATA/NCQ recovery to run in the atabus thread
 1.110.4.3 17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.110.4.2 17-Sep-2018  jdolecek move low-level protocol handlers hooks from ata_xfer to separate struct,
initialized statically

primarily to reduce ata_xfer struct size, but also improves readibility,
and enforces consistency
 1.110.4.1 31-Aug-2018  jdolecek refactor ata_xfer to be just dumb structure; move all callouts/condvars out

retry callout to wd(4); reset callout and the active/cmd finish condvars
to channel queue; change code using the condvars so it works if there
are multiple waiters

simplify the async wait code for cmds, replace ata_wait_xfer()/ata_wake_xfer()
with ata_wait_cmd()

fix the callout_invoking/ack race handling code for timeouts to
actually have chance to work; change mvsata(4) to use generic timeout func

towards resolution of kern/52614
 1.110.2.3 21-Apr-2020  martin Sync with HEAD
 1.110.2.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.110.2.1 10-Jun-2019  christos Sync with HEAD
 1.113.4.1 30-Dec-2022  martin Pull up following revision(s) (requested by tsutsui in ticket #1557):

sys/dev/ic/ahcisata_core.c: revision 1.83
sys/dev/ic/ahcisata_core.c: revision 1.102
sys/dev/ata/ata.c: revision 1.164
sys/dev/ata/ata_wdc.c: revision 1.115
sys/dev/ata/ata_recovery.c: revision 1.4
sys/dev/ic/siisata.c: revision 1.42
sys/dev/ic/wdc.c: revision 1.308
sys/dev/ic/mvsata.c: revision 1.56
sys/dev/scsipi/atapi_wdc.c: revision 1.138
sys/dev/ic/siisata.c: revision 1.49
sys/dev/ata/atavar.h: revision 1.105
sys/dev/ata/wd.c: revision 1.460
sys/dev/ata/ata.c: revision 1.155
sys/dev/ata/wd.c: revision 1.462
sys/dev/ata/atavar.h: revision 1.109
sys/dev/ata/satapmp_subr.c: revision 1.16
sys/dev/ic/wdc.c: revision 1.299
sys/dev/ic/ahcisata_core.c: revision 1.93
sys/dev/ata/ata_wdc.c: revision 1.120
sys/dev/ic/wdcvar.h: revision 1.100
sys/dev/scsipi/atapi_wdc.c: revision 1.141
sys/dev/ic/mvsata.c: revision 1.61
sys/dev/usb/umass_isdata.c (apply patch)

drop wd lock in wdstart1() before calling the ata_bio hook; when called
from ata thread context, that can still need to sleep for wdc attachments
in wdcwait()

fix use-after-free for ata xfer on bio submission found by KASAN
driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself
PR kern/55169 by Nick Hudson

Function declaration formating whitespace consistency. NFCI.

PR kern/56403
Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.
(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.
Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().
(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).
"Go ahead" by jdolecek@.
 1.114.2.1 20-Apr-2020  bouyer Sync with HEAD
 1.118.2.1 03-Jan-2021  thorpej Sync w/ HEAD.
 1.5 10-Oct-2017  jdolecek split off functions used by 'wd* at umass?' into separate file, unfortunately
the previous approach with NATABUS doesn't work for kernels which include
MODULAR, but not atabus - such as macppc and evbarm
 1.4 08-Oct-2017  jdolecek as a stopgap measure, include necessary parts of dev/ata/ata.c even
if no atabus is configured, so that kernels with just 'wd* at umass?'
and no atabus compile; fixes kernel builds for vax, pmax and evbmips
after NCQ branch merge

XXX wd* at umass? still actually untested after the merge, need to discuss
XXX what to do with it
 1.3 13-Feb-2009  apb Use "defopt MODULAR" in sys/conf/files, and #include "opt_modular.h"
in all kernel sources that use the MODULAR option.
Proposed in tech-kern on 18 Jan 2009.
 1.2 12-Nov-2008  ad branches: 1.2.4;
Remove LKMs and switch to the module framework, pass 1.

Proposed on tech-kern@.
 1.1 07-Sep-2006  itohy branches: 1.1.4; 1.1.6; 1.1.12; 1.1.58; 1.1.62; 1.1.68; 1.1.72;
Add PIOBM (busmastering transfer using ATA PIO mode) support.
The PIOBM is used by only one driver (will be added later,
stay tuned) and intruduce an attribute "ata_piobm" so that
it will be conditionally compiled in.
The "ata_dma" (busmastering transfer using ATA DMA mode) and
"ata_udma" (busmastering transfer using ATA Ultra DMA mode)
attributes are also added for consistency, but unused for now.
 1.1.72.2 03-Mar-2009  skrll Sync with HEAD.
 1.1.72.1 19-Jan-2009  skrll Sync with HEAD.
 1.1.68.1 13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.1.62.1 04-May-2009  yamt sync with head.
 1.1.58.1 17-Jan-2009  mjf Sync with HEAD.
 1.1.12.2 30-Dec-2006  yamt sync with head.
 1.1.12.1 07-Sep-2006  yamt file ataconf.h was added on branch yamt-lazymbuf on 2006-12-30 20:47:54 +0000
 1.1.6.2 14-Sep-2006  yamt sync with head.
 1.1.6.1 07-Sep-2006  yamt file ataconf.h was added on branch yamt-pdpolicy on 2006-09-14 12:31:27 +0000
 1.1.4.2 09-Sep-2006  rpaulo sync with head
 1.1.4.1 07-Sep-2006  rpaulo file ataconf.h was added on branch rpaulo-netinet-merge-pcb on 2006-09-09 02:49:44 +0000
 1.2.4.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.46 05-Jul-2022  andvar s/retrys/retries/ in comments.
 1.45 15-May-2020  jdolecek remove unused atacmd_tostatq()
 1.44 07-Oct-2017  jdolecek Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.43 30-Oct-2013  drochner branches: 1.43.18;
-recognize CF cards by the magic value in inquiry data
-kill CFG_ATAPI_MASK, didn't see anything in the specs supporting
that it exists
 1.42 09-Jan-2013  riastradh branches: 1.42.2;
Name the ATA SECURITY commands.
 1.41 19-Oct-2012  drochner Implement experimental support to pass notifications that a file
was deleted from the filesystem to the disk driver, commonly
known as "discard" or "trim".
fs/driver support is in ffs and ata wd for now.
This is what was posted here:
http://mail-index.netbsd.org/tech-kern/2012/02/28/msg012813.html
with minor cleanup, and the global switch replaced by a mount option.
 1.40 24-Oct-2011  jakllsch branches: 1.40.2; 1.40.12;
SATA_SIGNAL_GEN3 definition for Serial ATA capabilities word in
IDENTIFY DEVICE structure.
 1.39 24-Oct-2011  jakllsch Add defines for SET FEATURES for the SATA, PUIS and AAM feature sets.
 1.38 25-Jan-2010  jakllsch Add some more elements to struct ataparams.

These mostly concern the new Long Physical Sectors
and Long Logical Sectors feature sets.

Information from a draft of ACS-2, with some additional
in-between members and bits being borrowed from OpenBSD.

There's lots more in ACS-2 we could add, but for the most
part it would be superfluous.

XXX: Some elements marked as reserved no longer are,
and some elements we still reference are now obsolete.
 1.37 22-Jan-2010  jakllsch Correct an error in previous commit.
 1.36 21-Jan-2010  jakllsch Convert to C99 types. Make whitespace more consistent.
 1.35 10-Jan-2010  tsutsui Provide atacmd_to48() in case of #if defined(_STANDALONE) too
because standalone wdc driver in cobalt bootloader requires it
for LBA48 support.
 1.34 19-Oct-2009  bouyer Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !
 1.33 08-Jun-2009  jakllsch Add atap_max_lba member to ataparams for LBA48 Maximum Address. From OpenBSD.
While here, renumber some __reserved member names to be sequential.
Also, add a member for the World Wide Name.

Use atap_max_lba in wd.c instead of offsets into a __reserved member.
 1.32 25-Dec-2007  perry branches: 1.32.10; 1.32.20; 1.32.24; 1.32.28;
Convert many of the uses of __attribute__ to equivalent
__packed, __unused and __dead macros from cdefs.h
 1.31 18-Nov-2007  christos branches: 1.31.2; 1.31.6;
From Marco Trillo (marcotrillo at gmail dot com):
Add Advanced power management in atactl(8)
 1.30 21-Aug-2007  bouyer branches: 1.30.2; 1.30.6; 1.30.8;
Improved SATA support, from Jonathan A. Kollasch in PR bin/36772:
* use full 5-bit command queue depth that SATA supports
* decode SATA bits
 1.29 16-Nov-2006  christos branches: 1.29.8; 1.29.18; 1.29.22;
__unused removal on arguments; approved by core.
 1.28 30-Sep-2006  itohy A little effort against kernel bloat....
Exclude ATA DMA support if no ATA DMA capable drivers are compiled in.
 1.27 24-Sep-2006  xtraeme Typo: WCDD_SECURITY_FREEZE -> WDCC_SECURITY_FREEZE

Noticed by Sergey Svishchev.
 1.26 16-Feb-2006  perry branches: 1.26.14; 1.26.16;
Change "inline" back to "__inline" in .h files -- C99 is still too
new, and some apps compile things in C89 mode. C89 keywords stay.

As per core@.
 1.25 24-Dec-2005  perry branches: 1.25.2; 1.25.4; 1.25.6;
Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.
 1.24 11-Dec-2005  christos merge ktrace-lwp.
 1.23 12-Apr-2005  drochner branches: 1.23.2;
add definition for "security freeze"
 1.22 27-Feb-2005  perry branches: 1.22.2;
nuke trailing whitespace
 1.21 23-Aug-2004  he branches: 1.21.4; 1.21.6;
Put an #ifdef _KERNEL around inline functions using panic().
Fixes compile problem for the atactl program for the vax target.
 1.20 21-Aug-2004  thorpej - Move to48() to atareg.h and rename it atacmd_to48().
- Add an atacmd_tosataq() that converts a 32-bit command to a Native
SATA Queued command.
 1.19 12-Aug-2004  thorpej Add SATA queued read/write commands.
 1.18 03-May-2004  bouyer When using LBA48, use Flush cache extended command if supported.
From Christos Zoulas in PR 25366, with minor adjustements from me.
 1.17 14-Mar-2004  wiz branches: 1.17.2;
It's extension, not extention. From Miod Vallat (miod at online fr).
 1.16 30-Dec-2003  thorpej Move the register definitions for ATA Task File registers to atareg.h;
they are also used by controllers that do not map the registers into the
system address space, and are thus distinct from the "controller" registers.
 1.15 30-Dec-2003  thorpej SMART info is logically part of the register set; move it from atavar.h
to atareg.h
 1.14 30-Nov-2003  yamt add definitions of new feature bits.
 1.13 30-Nov-2003  yamt - rename WDC_CAP_LBA48 to ATA_CMD2_LBA48.
- remove WDC_CAP_QUEUE. (there's already ATA_CMD2_RWQ.)

as discussed on tech-kern@.
 1.12 26-Nov-2003  yamt remove ATA_CMD2_SMART, which seems wrong and isn't used.
 1.11 02-Nov-2003  wiz Fix two typos. From Tom Cosgrove via jmc@openbsd.
 1.10 23-Sep-2003  skd Additional definitions as per SATA specifications.
Approved by Manuel Bouyer.
 1.9 05-Aug-2002  soren branches: 1.9.6;
Add ATA SMART reporting support from Ben Collver in kern/12787.
 1.8 13-Jan-2002  christos branches: 1.8.8; 1.8.10;
Add LBA48 support based on patches from Keisuke YOSHIDA. KNF a bit while
I am there.
 1.7 05-Jul-2001  toshii branches: 1.7.2;
Fix typo. s/extention/extension/
 1.6 29-Mar-2000  augustss branches: 1.6.6;
Fix a couple of misspelled comments.
 1.5 18-Jan-1999  bouyer branches: 1.5.2; 1.5.8;
move wd.c:print_wderror() to ata.c:ata_perror().
In wdc_probe_caps() add code to guess the ATA revision supported (if
ATA4 if Ultra-DMA, ATA2 if PIO mode > 2). We can't rely on param.atap_ata_major
here, at last one Ultra-DMA drive claims to support only ATA-3.
Use the ATA version in ata_perror(), and to try a flush cache command
in a shutdown hook for IDE drives.
 1.4 01-Dec-1998  tsubai struct ataparam has endian dependence, so add big-endian case.
 1.3 18-Nov-1998  kenh Don't protect this with #ifdef _KERNEL
 1.2 12-Oct-1998  bouyer Merge bouyer-ide
 1.1 04-Jun-1998  bouyer branches: 1.1.2;
file atareg.h was initially added on branch bouyer-ide.
 1.1.2.1 04-Jun-1998  bouyer Commit changes to the IDE system in a branch. This allows a better separation
between higth-level and low-level (i.e. registers read/write) and generalize
the queue for all commands. This also add supports for IDE DMA.
 1.5.8.1 20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
A i386 GENERIC kernel compiles without the siop, ahc and bha drivers
(will be updated later). i386 IDE/ATAPI and ncr work, as well as
sparc/esp_sbus. alpha should work as well (untested yet).
siop, ahc and bha will be updated once I've updated the branch to current
-current, as well as machine-dependant code.
 1.5.2.1 07-Jul-2000  he Apply patch (requested by bouyer):
Add support for the following PCIIDE controllers:
o AMD 756
o CMD PCI0648 and PCI0649
o Hightpoint HPT366
o OPTi 82c621 (and a few of its derivatives)
o Promise Ultra/33 and Ultra/66
o Intel 82801 (ICH/ICH0)
Also fix PR#10437 (detect more ATAPI devices).
 1.6.6.3 13-Aug-2002  nathanw Catch up to -current.
 1.6.6.2 28-Feb-2002  nathanw Catch up to -current.
 1.6.6.1 24-Aug-2001  nathanw Catch up with -current.
 1.7.2.2 06-Sep-2002  jdolecek sync kqueue branch with HEAD
 1.7.2.1 11-Feb-2002  jdolecek Sync w/ -current.
 1.8.10.1 14-May-2004  jdc Pull up revision 1.14 via patch (requested by bouyer in ticket #1690)

add definitions of new feature bits.
 1.8.8.1 29-Aug-2002  gehenna catch up with -current.
 1.9.6.7 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.9.6.6 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.9.6.5 21-Sep-2004  skrll Fix the sync with head I botched.
 1.9.6.4 18-Sep-2004  skrll Sync with HEAD.
 1.9.6.3 25-Aug-2004  skrll Sync with HEAD.
 1.9.6.2 12-Aug-2004  skrll Sync with HEAD.
 1.9.6.1 03-Aug-2004  skrll Sync with HEAD
 1.17.2.1 09-May-2004  jdc Pull up revision 1.18 (requested by bouyer in ticket #262)

When using LBA48, use Flush cache extended command if supported.
From Christos Zoulas in PR 25366, with minor adjustements from me.
 1.21.6.1 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.21.4.1 29-Apr-2005  kent sync with -current
 1.22.2.1 15-Apr-2005  tron Pull up revision 1.23 (requested by drochner in ticket #156):
add definition for "security freeze"
 1.23.2.4 21-Jan-2008  yamt sync with head
 1.23.2.3 07-Dec-2007  yamt sync with head
 1.23.2.2 03-Sep-2007  yamt sync with head.
 1.23.2.1 30-Dec-2006  yamt sync with head.
 1.25.6.1 22-Apr-2006  simonb Sync with head.
 1.25.4.1 09-Sep-2006  rpaulo sync with head
 1.25.2.1 18-Feb-2006  yamt sync with head.
 1.26.16.2 10-Dec-2006  yamt sync with head.
 1.26.16.1 22-Oct-2006  yamt sync with head
 1.26.14.1 18-Nov-2006  ad Sync with head.
 1.29.22.2 21-Nov-2007  joerg Sync with HEAD.
 1.29.22.1 03-Sep-2007  jmcneill Sync with HEAD.
 1.29.18.1 03-Sep-2007  skrll Sync with HEAD.
 1.29.8.1 09-Oct-2007  ad Sync with head.
 1.30.8.2 18-Feb-2008  mjf Sync with HEAD.
 1.30.8.1 19-Nov-2007  mjf Sync with HEAD.
 1.30.6.1 18-Nov-2007  bouyer Sync with HEAD
 1.30.2.1 09-Jan-2008  matt sync with HEAD
 1.31.6.1 02-Jan-2008  bouyer Sync with HEAD
 1.31.2.1 26-Dec-2007  ad Sync with head.
 1.32.28.1 21-Apr-2010  matt sync to netbsd-5
 1.32.24.1 23-Jul-2009  jym Sync with HEAD.
 1.32.20.1 10-Jan-2010  snj Pull up following revision(s) (requested by dsl in ticket #1231):
sys/dev/ata/atareg.h: revision 1.33
sys/dev/ata/wd.c: revision 1.378
Add atap_max_lba member to ataparams for LBA48 Maximum Address. From OpenBSD.
While here, renumber some __reserved member names to be sequential.
Also, add a member for the World Wide Name.
Use atap_max_lba in wd.c instead of offsets into a __reserved member.
 1.32.10.2 11-Mar-2010  yamt sync with head
 1.32.10.1 20-Jun-2009  yamt sync with head
 1.40.12.5 03-Dec-2017  jdolecek update from HEAD
 1.40.12.4 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.40.12.3 25-Feb-2013  tls resync with head
 1.40.12.2 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.40.12.1 09-Oct-2012  bouyer Support transfers of up to MACHINE_MAXPHYS in all pciide variants, and ahci.
wd(4) limits its maxphys depending on the drives's capability (64k sectors
for LBA48, 256 sectors for LBA and 128 sectors for older devices).

I assumed all pciide controllers could do MACHINE_MAXPHYS transfers, but
this may not be true. The capabilities of each controller variants should be
looked at more closely.
 1.40.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.40.2.2 23-Jan-2013  yamt sync with head
 1.40.2.1 30-Oct-2012  yamt sync with head
 1.42.2.1 18-May-2014  rmind sync with head
 1.43.18.3 19-Jul-2017  jdolecek tighen and expand error handling, mostly for NCQ use cases:
- make retry timeout callout per xfer, i.e. retry separately
- zero whole bio struct on retry to avoid more stale state
- add a REQUEUE option, which doesn't bump retry count
- add ata_read_log_ext_ncq() for NCQ recovery
- adjust logic for activating xfers - allow next command only when
it's for same drive, several concurrent are only supported when HBA
and driver support FIS-based switching
- add new ata_timeout() which handles race between callout_stop()
and the invokation, add appropriate handling on deactivate/free paths
- stop using ch_status/ch_error in non-wdc code; later it will be dropped
completely
 1.43.18.2 24-Apr-2017  jdolecek set NCQ priority field to 'high' for BPRIO_TIMECRITICAL transfers if drive
supports it
 1.43.18.1 19-Apr-2017  jdolecek add ATA FUA support
 1.109 05-Oct-2021  rin PR kern/56403

Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.

(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.

Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().

(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).

"Go ahead" by jdolecek@.
 1.108 25-May-2020  jdolecek make ata_downgrade_mode() static, it's not used anywhere else
 1.107 02-May-2020  thorpej Back out changes to use a threadpool for now; it's causing trouble
for some folks on Thinkpads.
 1.106 25-Apr-2020  thorpej Rather than creating a kthread-per-channel, use a threadpool and a
threadpool-job-per-channel for the in-thread-context work that needs
to be done (which is rare).

On one of my test systems, this results in the total number of LWPs
after multi-user boot dropping from 116 to 78.
 1.105 13-Apr-2020  jdolecek fix use-after-free for ata xfer on bio submission found by KASAN

driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself

PR kern/55169 by Nick Hudson
 1.104 04-Apr-2020  jdolecek branches: 1.104.2;
fix deadlock in wdcwait() when xfer timeout happens while the atabus
thread sleeps in wdcwait() - check current lwp rather than relying
on global ATACH_TH_RUN channel flag

should fix the hang part of the problem reported in
http://mail-index.netbsd.org/netbsd-users/2020/03/12/msg024249.html

thanks to Paul Ripke for providing extensive debugging info
 1.103 05-Apr-2019  bouyer branches: 1.103.4;
drop AT_RST_NOCMD, it's a cut'n'paste side effect
 1.102 05-Apr-2019  christos unbreak the build, someone forgot to commit more stuff :-)
 1.101 24-Oct-2018  jdolecek detach the controller itself on shutdown; adjust to not detach already
detached atabus/channel
 1.100 22-Oct-2018  jdolecek Merge jdolecek-ncqfixes branch

- ata_xfer's are dynamicall allocated as needed using a pool, no longer
limited to number of possible openings supported by controller; dump
and recovery paths use dedicated pre-allocated storage
- moved callouts and condvars from ata_xfer to queue or channel, so that
ata_xfer does not need special initialization
- slot allocation now done when xfer is being activated, uncoupled
from memory allocation; active slots are no longer tracked by controller
code
- channel and drive reset is done always via the atabus thread, and
now executes with channel locked the whole time
- NCQ recovery moved to shared function, and run via the thread also
- added some workarounds for buggy error recovery AHCI emulation in QEMU
and Parallels

designed to primarily fix kern/52614, but might also help with kern/47041
and kern/53183
 1.99 10-Aug-2018  jdolecek branches: 1.99.2;
fix race in wd_lastclose() on systems with two ide disks on same
channel, which happened when one disk had pending I/O while the other
disk executed the final disk flush - need to restart bufq processing
once xfer is freed in this case

it could happen e.g. on boot when system executes fsck on different
partitions on the two drives in parallell and hence open and closes
the disk devices repeatedly

add KASSERT() for empty bufq on wd_lastclose(), and fix similar issue
also on suspend/standby path

this was introduced by the NCQ merge and not dksubr - before the merge
each drive had their own xfer, so they could not block each other

fixes PR kern/52783 by Onno van der Linden; many thanks for extensive
help with tracking this down
 1.98 06-Aug-2018  jdolecek add wddebug() which dumps some status for attached disks; indended for
debugging of PR kern/52783
 1.97 01-Jun-2018  macallan branches: 1.97.2;
add a flag to start DMA before issuing commands - needed to work around a bug
in some SATA chips which get confused if the disk responds too fast
Mostly for K2 SATA / svwsata found in G5 Macs
adapted from OpenBSD
 1.96 16-Apr-2018  jdolecek remove superfluous semicolon
 1.95 17-Oct-2017  jdolecek branches: 1.95.2;
reintroduce ATACH_IRQ_WAIT flag for attachments using wdcintr(), only
process the interrupt when the flag is set - this fixes spurious interrupt
during post-reset drive setup in wdc_ata_bio_start(), and wdc_atapi_start()

while those functions set WDCTL_IDS, this seems to be ignored by certain
(maybe all) PCI-IDE controllers; usually the implicit KERNEL_LOCK() would
prevent the interrupt anyway, but not when the start routine is started
from the atabus thread, which doesn't take it

fixes 'panic: wdc_ata_bio_intr: bad state' reported on current-users
by Chavdar Ivanov
 1.94 10-Oct-2017  jdolecek split off functions used by 'wd* at umass?' into separate file, unfortunately
the previous approach with NATABUS doesn't work for kernels which include
MODULAR, but not atabus - such as macppc and evbarm
 1.93 07-Oct-2017  jdolecek Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.92 10-Sep-2014  matt branches: 1.92.8;
g/u_int[0-9]*_t/ s/u_int/uint/g
 1.91 03-Apr-2013  bouyer Fix kernel dump on ahci controller, by making sure we won't sleep
while dumping:
- introduce ata_delay() which either use delay() or kpause()
depending on flags. use it in sata_reset_interface() and
some ahci functions
- kill ATA_NOSLEEP, it was tested but never set. use ATA_POLL instead.
- reduce delay while polling in ahci, to speed up the dump

Should fix PR kern/41095
 1.90 31-Jul-2012  bouyer branches: 1.90.2;
Apply back changes that were reverted on Jul 24 and Jul 26 (general ata/wdc
cleanup and SATA PMP support), now that I'm back to fix the fallouts.
 1.89 29-Jul-2012  jakllsch Remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas. Originally part of a commit by bouyer@.
 1.88 26-Jul-2012  jakllsch Revert, with intention of restoring in a less invasive way, the SATA Port
Multiplier code.

ok christos@
 1.87 24-Jul-2012  jakllsch Revert dsl@'s changes of Sun, 15 Jul 2012 10:55:35 +0000 and
Sun, 15 Jul 2012 10:56:50 +0000, excepting the kernel version bump.
First step in reverting regressions to ata(4) subsystem during the addition of
port multiplier support.
 1.86 15-Jul-2012  dsl Some namespace protection (and add greppablity).
Prefix the DRIVE_ and DRIVET_ constants from atavar.h with ATA_.
Don't use an enum for drive_type - you don't know how big it will be.
Move driver_type to avoid implicit structure padding (esp on arm).
This change is purely lexical and mechanical.

Update to 6.99.9 - this wasn't done when the SATA PMP changes
were made - I'm sure they warranted a bump.
 1.85 02-Jul-2012  bouyer Add sata Port MultiPlier (PMP) support to the ata bus layer,
as described in
http://mail-index.netbsd.org/tech-kern/2012/06/23/msg013442.html
PMP support in integrated to the atabus layer.
struct ata_channel's ch_drive[] is not dynamically allocated, and ch_ndrive
(renamed to ch_ndrives) closely reflects the size of the ch_drive[] array.
Add helper functions atabus_alloc_drives() and atabus_free_drives()
to manage ch_drive[]/ch_ndrives.
Add wdc_maxdrives to struct wdc_softc so that bus front-end can specify
how much drive they really support (master/slave or single).
ata_reset_drive() callback gains a uint32_t *sigp argument which,
when not NULL, will contain the signature of the device being reset.
While there, some cosmetic changes:
- added a drive_type enum to ata_drive_datas, and stop encoding the
probed drive type in drive_flags (we were out of drive flags anyway).
- rename DRIVE_ATAPIST to DRIVE_ATAPIDSCW to better reflect what this
really is
- remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas
- factor out the interpretation of SATA signatures in sata_interpet_sig()

propagate these changes to the ATA HBA drivers, and add support for PMP
to ahcisata(4) and siisata(4).

Thanks to:
- Protocase (http://www.protocase.com/) which provided a system
with lots of controllers, SATA PMP and drive slots
- Conservation Genomics Laboratory, Department of Biology, New Mexico State
University for hosting the above system
- Brook Milligan, who set up remote access and has been very responsive
when SATA cable move was needed
 1.84 24-Jan-2012  jakllsch Rework struct ata_command to support LBA28 and LBA48-protocol commands.
Implement 28- and 48-bit command issuance and parameter read-back in the
various ATA host drivers. Add LBA28-protocol support to ATAIOCCOMMAND ioctl.
 1.83 29-Oct-2011  jakllsch branches: 1.83.2; 1.83.6;
Some comment improvements; spelling and grammar.
 1.82 30-Apr-2011  jakllsch Add simplistic atabus(4) rescan support.
 1.81 18-Apr-2011  rmind Replace few simple_lock and ltsleep/wakeup uses with mutex(9) and condvar(9).

Note to all: please replace old primitives in your code! Thanks.
 1.80 19-Oct-2009  bouyer branches: 1.80.4; 1.80.6;
Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !
 1.79 12-May-2009  cegger struct device * -> device_t, no functional changes intended.
 1.78 16-Nov-2008  bouyer branches: 1.78.4;
cpu_intr_p() doesn't account for software interrupts (e.g. callouts) so
we can't use it here. Rssurect ATACH_TH_RUN, backing out
src/sys/dev/ata/ata.c 1.101
src/sys/dev/ata/ata_wdc.c 1.90
src/sys/dev/ata/atavar.h 1.77
src/sys/dev/ic/wdc.c 1.255
src/sys/dev/scsipi/atapi_wdc.c 1.108
Should fix kern/39927 and kern/39725.
 1.77 02-Oct-2008  bouyer branches: 1.77.2; 1.77.4;
Kill ATACH_TH_RUN and use cpu_intr_p() instead.
 1.76 18-Mar-2008  cube branches: 1.76.4; 1.76.6; 1.76.10;
Split device_t and softc for ATA devices, as well as wd(4). Other
cosmetic changes where appropriate.
 1.75 09-Dec-2007  jmcneill branches: 1.75.6; 1.75.10;
Merge jmcneill-pm branch.
 1.74 09-Jul-2007  ad branches: 1.74.6; 1.74.8; 1.74.16; 1.74.18;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements
 1.73 30-Sep-2006  itohy branches: 1.73.8; 1.73.10;
A little effort against kernel bloat....
Exclude ATA DMA support if no ATA DMA capable drivers are compiled in.
 1.72 07-Sep-2006  itohy branches: 1.72.2; 1.72.4;
Add PIOBM (busmastering transfer using ATA PIO mode) support.
The PIOBM is used by only one driver (will be added later,
stay tuned) and intruduce an attribute "ata_piobm" so that
it will be conditionally compiled in.
The "ata_dma" (busmastering transfer using ATA DMA mode) and
"ata_udma" (busmastering transfer using ATA Ultra DMA mode)
attributes are also added for consistency, but unused for now.
 1.71 24-Dec-2005  perry branches: 1.71.4; 1.71.8;
Remove leading __ from __(const|inline|signed|volatile) -- it is obsolete.
 1.70 11-Dec-2005  christos merge ktrace-lwp.
 1.69 07-Jun-2005  peter branches: 1.69.2;
Fix typos.
 1.68 16-May-2005  bouyer Add a ata_queue_idle() function, which freeze a queue and tsleep() until the
controller is idle.
Change the powerhook function to call ata_queue_idle() on standby/suspend,
and ata_reset_channel() on resume (to wake up the disks and start from a
clean state).
Fix PR kern/30194 by Lubomir Sedlacik.
 1.67 27-Feb-2005  perry branches: 1.67.2;
nuke trailing whitespace
 1.66 26-Jan-2005  jmcneill Add powerhook for atabus.
 1.65 30-Oct-2004  bouyer branches: 1.65.4; 1.65.6;
Allow polled commands to be queued, if ata_exec_xfer() is called in thread
context. For this introduce 3 new xfer tags:
- C_WAIT, equivalent of AT_WAIT
- C_WAITACT, when there is a thread waiting on this xfer to become the active
one (that is, to be at the head of the queue)
- C_FREE, set by ata_free_xfer() when it can't free the xfer because it's
still in use. The holder should then free the xfer ASAP.

If ata_exec_xfer() is called with (C_POLL | C_WAIT), and there is already
xfers to be processed, assert C_WAITACT and sleep.
atastart() checks for C_WAITACT, and wakeup the thread waiting for this xfer
to become active if set. atastart() won't process this xfer, it's the
responsability of the thread waked up to handle it.

Fix (the right way) kern/27421 by Martin Husemann.
 1.64 20-Aug-2004  thorpej - Add an (*ata_reset_channel)() member to ata_bustype.
- Add an ata_reset_channel() function that performs the common parts
of resetting an ATA channel, which uses the (*ata_reset_channel)()
callback to do the heavy lifting. Adjust callers to use ata_reset_channel()
instead of wdc_reset_channel().

This removes the last wdc-specific code from ata.c!
 1.63 20-Aug-2004  thorpej Move common parts of attaching an ata_channel to a new ata_channel_attach()
function.
 1.62 20-Aug-2004  thorpej - Add an atac_bustype_ata member to atac_softc to hold a pointer to
the ata_bustype for the controller. Fill it in and use it instead
of referencing wdc_ata_bustype directly.
- Add an atac_atapibus_attach member to atac_softc to hold a pointer
to the function that attaches the ATAPI bus for the controller. Fill
it in and use it instead of referencing wdc_atapibus_attach directly.
 1.61 20-Aug-2004  thorpej Rename the ata_bustype "ata_reset_channel" member to "ata_reset_drive"
to reflect the new reality after Manuel added the bus reset ioctls.
 1.60 20-Aug-2004  thorpej Move most of wdc_softc into a new atac_softc structure that contains
info common to all types of ATA controllers.
 1.59 14-Aug-2004  thorpej - Split the register handles out of struct wdc_channel into a separate
wdc_regs structure, and array of which (indexed per channel) is pointed
to by struct wdc_softc.
- Move the resulting wdc_channel structure to atavar.h and rename it to
ata_channel. Rename the corresponding flags.
- Add a "ch_ndrive" member to struct ata_channel, which indicates the
maximum number of drives that can be present on the channel. For now,
this is always 2. Add an ATA_MAXDRIVES constant that places an upper
limit on this value, also currently 2.
 1.58 13-Aug-2004  thorpej Move wdcstart() to ata.c and rename it to atastart().
 1.57 13-Aug-2004  thorpej Move wdc_exec_xfer() to ata.c and rename it ata_exec_xfer().
 1.56 12-Aug-2004  thorpej Move wdc_addref() and wdc_delref() to ata.c and rename them to
ata_*.
 1.55 12-Aug-2004  thorpej Move wdc_kill_pending() to ata.c and rename it ata_kill_pending().
 1.54 12-Aug-2004  thorpej - Move wdc_xfer_pool, wdc_get_xfer(), wdc_free_xfer() to ata.c, and rename
to ata_*.
- Use a static initializer for the ata_xfer_pool.
 1.53 12-Aug-2004  thorpej Move wdc_probe_caps() to ata.c and rename it ata_probe_caps().
 1.52 12-Aug-2004  thorpej Move wdc_downgrade_mode() to ata.c and rename it to ata_downgrade_mode().
 1.51 12-Aug-2004  thorpej Move wdc_print_modes() into ata.c and rename it ata_print_modes().
 1.50 12-Aug-2004  thorpej Rename some constants:
WDC_COMPLETE -> ATACMD_COMPLETE
WDC_QUEUED -> ATACMD_QUEUED
WDC_TRY_AGAIN -> ATACMD_TRY_AGAIN
 1.49 12-Aug-2004  thorpej Rename "struct wdc_command" to "struct ata_command".
 1.48 11-Aug-2004  mycroft Add two helper functions -- wdc_datain_pio() and wdc_dataout_pio() -- which
encapsulate the logic for the various methods of transferring data. Use these
throughout.
 1.47 04-Aug-2004  bouyer Change wdc_kill_pending() to take a struct ata_drive_datas * as argument,
and kill only pending requests for this drive.
Implement a DRIVE_WAITDRAIN flag, which will cause the active command to
be killed once complete.
Other minor fixes.
Now it's possible to detach a ATA or ATAPI device from ioctl even when
a dd on the raw char partition is running.
 1.46 04-Aug-2004  bouyer Add
struct ata_xfer *active_xfer
to ata_queue. Now the active xfer isn't the head of the queue any more,
this makes a few things easier (this will also help for tagged queuing
support).
Remove the WDCF_ACTIVE flag, test active_xfer != NULL instead.
clean up wdc_free_xfer() and kill_xfer().
Clean up wdc_reset_channel(), and make it issue a ATAPI_SOFT_RESET if the
active command is ATAPI.
In wdc_atapi_get_params(), use AT_WAIT | AT_POLL for ATAPI_SOFT_RESET,
so that we'll use tsleep() instead of delay().
In wdc_atapi_start(), call wdc_dmawait() at the right place.
 1.45 01-Aug-2004  bouyer Implement an atabus control device, and define some ATA bus control
IOCTLS. Implement ATABUSIORESET, which will reset the given ATA bus.
 1.44 31-Jul-2004  bouyer Implement asynchronous channel reset.
Use this to reset the channel before doing a dump, instead of the hack in
wdc_exec_xfer() based on C_POLL. This hack was causing problems on
controllers with a shared queue, because we now can have C_POLL set during
concurent channels probes (problem found and analysed on sparc64 by
Martin Husemann).
This should even make core dumps marginally more reliable on ATA drives.
 1.43 01-Jun-2004  mycroft Only downgrade modes due to an actual CRC error. Downgrading on other errors
is anti-social -- especially given that there's no way to upgrade again short
of rebooting.

Also, downgrade UDMA modes more slowly. It's entirely possible that they're
using an 80-wire cable, but it's just too long for the higher modes, or there
is minor crosstalk.
 1.42 27-May-2004  thorpej Clean-up of use of "precomp". It is used as "features" everywhere, so
rename the fields in wdc_command as appropriate.
 1.41 14-Apr-2004  minoura Change the value of DRIVE_ATAPIST which was same as that of DRIVE_DMAERR.
Ok'd by bouyer.
 1.40 03-Jan-2004  thorpej branches: 1.40.2;
Rename "struct channel_softc" to "struct wdc_channel".
 1.39 01-Jan-2004  thorpej Rename:
- wdc_xfer to ata_xfer
- channel_queue to ata_queue
and move them to <dev/ata/atavar.h> so they can be used by non-wdc ATA
controllers. Clean up the member names of these structures while at it.
 1.38 30-Dec-2003  thorpej SMART info is logically part of the register set; move it from atavar.h
to atareg.h
 1.37 30-Dec-2003  thorpej Move most of the atabus layer into ata.c.
 1.36 20-Dec-2003  lha add more fields to ata_smart_attr, add structures for selftest log
 1.35 14-Dec-2003  thorpej Instead of calling wddone() directly, set a (*drv_done)() callback in
the ata_drive_datas structure and invoke that.
 1.34 14-Dec-2003  thorpej Move struct ata_device from wdvar.h to atavar.h.
 1.33 14-Dec-2003  thorpej Move struct ata_bustype from wdvar.h to atavar.h.
 1.32 14-Dec-2003  thorpej Move struct ata_bio from wdvar.h to atavar.h.
 1.31 14-Dec-2003  thorpej Tidy up the whitespace.
 1.30 14-Dec-2003  thorpej De-__P and use ANSI-style function decls.
 1.29 08-Oct-2003  bouyer Make the ATA mid-layer appears as atabus, as proposed in
http://mail-index.netbsd.org/tech-kern/2003/09/25/0006.html
This adds a device (atabus) between IDE controllers and wd or atapibus, to
have each ATA channel show up in the device tree. Later there will be atabus
devices in /dev, so that we can do IOCTL on them.
Each atabus has its own kernel thread, to handle operations that needs polling,
e.g. reset and others.

Device probing on each bus it defered to the atabus thread creation.
This allows to do the reset and basic device probes in parallel, which reduce
boot time on systems with several pciide controllers.
 1.28 05-Oct-2003  bouyer Remove references to University of California from my copyright notices.
 1.27 08-Jul-2003  itojun function prototype must not have variable name
 1.26 05-Aug-2002  soren branches: 1.26.6;
Add ATA SMART reporting support from Ben Collver in kern/12787.
 1.25 23-Apr-2002  bouyer branches: 1.25.2; 1.25.4;
More copyright fixes, pointed out by Thomas. Thanks !
 1.24 09-Apr-2002  bouyer In wdc_exec_command(), for data commands, read the status register after
the data transfer. This is mandatory for data out commands (although none are
used for now), and not forbiddend for data in commands. Also record if we
did transfers any data.
May solve kern/16159 by making the probe more robust in face of fake identify.
 1.23 03-Dec-2001  bouyer Update my copyrigth.
 1.22 03-Dec-2001  bouyer Allow wd to attach to something else than wdc/pciide (like USB<->ATA bridges):
- move some functions from ata.c to ata_wdc.c or wdc.c.
- add callbacks to struct ata_bustype so that wd.c doesn't call directly
functions from the lower level driver.
 1.21 02-Dec-2001  bouyer Clean up attach of wd/atapibus:
kill ata_atapi_attach. Change atapibus to use a struct scsipi_channel instead
of ata_atapi_attach as attach arch. Create a ata_device, compatible with
scsipi_channel, to attach wd.
 1.20 01-Dec-2001  bouyer For ATAPI tape drives, poll for DSC (using a callout), to get the real
status of the command, and make sure the drive is ready for the next one.
 1.19 25-Apr-2001  bouyer branches: 1.19.2;
Pull up the thorpej_scsipi branch to main branch.
This is a completely rewritten scsipi_xfer execution engine, and the
associated changes to HBA drivers. Overview of changes & features:
- All xfers are queued in the mid-layer, rather than doing so in an
ad-hoc fashion in individual adapter drivers.
- Adapter/channel resource management in the mid-layer, avoids even trying
to start running an xfer if the adapter/channel doesn't have the resources.
- Better communication between the mid-layer and the adapters.
- Asynchronous event notification mechanism from adapter to mid-layer and
peripherals.
- Better peripheral queue management: freeze/thaw, sorted requeueing during
recovery, etc.
- Clean separation of peripherals, adapters, and adapter channels (no more
scsipi_link).
- Kernel thread for each scsipi_channel makes error recovery much easier
(no more dealing with interrupt context when recovering from an error).
- Mid-layer support for tagged queueing: commands can have the tag type
set explicitly, tag IDs are allocated in the mid-layer (thus eliminating
the need to use buggy tag ID allocation schemes in many adapter drivers).
- support for QUEUE FULL and CHECK CONDITION status in mid-layer; the command
will be requeued, or a REQUEST SENSE will be sent as appropriate.

Just before the merge syssrc has been tagged with thorpej_scsipi_beforemerge
 1.18 15-May-2000  bouyer branches: 1.18.4; 1.18.6;
Sync my copyrigth notice
 1.17 02-Apr-2000  augustss Change the initial field in struct ata_atapi_attach and struct scsipi_link
slightly to allow scsibus and atapibus to attach to the same device.
Furthermore, only attach a scsibus when the bus type is BUS_SCSI.
 1.16 01-Apr-2000  bouyer - DMA code cleanup: pciide_dma_finish() doesn't stop/unload the current DMA op
if an IRQ was not detected, unless the force flag was given. Use this to
detect if the IRQ was for us (closer to shared IRQ for controllers which
don't have their own IRQ handler in pciide.c) and to poll for DMA xfer.
Also makes the timeout recovery code simpler.
- ATAPI cleanup: don't call controller-specific functions from atapiconf.c
(wdc_*), so that it's possible to attach an atapibus to something else
than a wdc/pciide (Hi Lennart :).
Overload struct scsi_adapter with struct atapi_adapter, defined
as struct scsi_adapter + atapi-specific callbacks. scsipi_link still points
to an scsi_adapter, atapi code casts it to atapi_adapter if needed.
Move atapi_softc to atapiconf.h so that it can be used by the underlying
controller code (e.g. atapi_wdc.c).
Add an atapi-specific callback *atapi_probedev(), which probe a drive
in a controller-specific way, allocate the sc_link and fills in the
ataparams if needed. It then calls atapi_probedev() (from atapiconf.c)
to do the generic initialisations and attach the device.
- While I'm there merge and centralise the state definitions in atavar.h.
It should now be possible to use a common ata/atapi routine to set the
drive's modes (will do later).
 1.15 17-Jan-2000  bouyer Improve the downgrade logic:
- If UDMA 2 is failing try UDMA 1 first, it helps in some cases
- downgrade if we get an error in the first 4000 xfers, or if we get
4 errors in 4000 xfers if the first 4000 went without troubles.

While I'm there commit a local change I have since some time to get my CD
probed: issue a "blanck" IDENTIFY before the one used to detect slave ghosts,
with my drive the first IDENTIFY following a controller reset fails with an
aborted command ...
 1.14 20-Oct-1999  enami Cancel active transfers on aic/wdc detach.
Also makes LS-120 drive works for me again.
 1.13 10-Mar-1999  bouyer branches: 1.13.2; 1.13.8; 1.13.10; 1.13.12;
Add a new disk flag: DRIVE_OLD for pre-ATA disks. probe routine will now set
DRIVE_OLD, DRIVE_ATA or DRIVE_ATAPI based on register signatures.
The attach routine will issue a IDENTIFY command for ATA/ATAPI disk,
to detect flase matches by the probe routine.
probe/attach should now be fully compliant with ata-4/ata-5. As a side
effect, ATAPI drives which improperly use ATA register signatures should now
be attached as ATAPI.
 1.12 07-Mar-1999  bouyer Keep track of CRC errors in Ultra-DMA mode. If we noticed a CRC error and we
need to downgrade, downgrade to PIO, as it has been shown if we got CRC errors
in Ultra-DMA mode, we will have silent data corruption in multiword DMA mode
(isn't IDE wonderfull ? :).
Set timeout to 1s for "normal" ata I/O, to minimise the effects of missed
interrupts.
 1.11 29-Jan-1999  bouyer Cosmetic changes, from Soren S. Jorvan.
 1.10 18-Jan-1999  bouyer move wd.c:print_wderror() to ata.c:ata_perror().
In wdc_probe_caps() add code to guess the ATA revision supported (if
ATA4 if Ultra-DMA, ATA2 if PIO mode > 2). We can't rely on param.atap_ata_major
here, at last one Ultra-DMA drive claims to support only ATA-3.
Use the ATA version in ata_perror(), and to try a flush cache command
in a shutdown hook for IDE drives.
 1.9 16-Dec-1998  bouyer Keep track of DMA errors, and downgrade the transert mode (UDMA ->DMA,
DMA->PIO) in case of 2 consecutive errors. Don't downgrade if the
PIO/DMA/UDMA modes were forced by a config flag.
 1.8 04-Dec-1998  bouyer AT_ERROR should be different from AT_TIMEOUT. pointed out by Matthias Drochner.
 1.7 02-Dec-1998  bouyer - change drive_flags from u_int8_t to u_int16_t
- keep the modes supported by the drive in struct ata_drive_datas (will be
later used for downgrading the DMA/PIO mode on error)
- use config flags to force/disable PIO/DMA/UDMA modes
- For the CMD PCI0643/6 setup DMA mode to DMA Read multiple.
 1.6 23-Nov-1998  kenh Add a new flag to wdc_command: ATA_READREG. This will indicate that all
device registers should be read back into the wdc_command structure after
successfull command completion. Use this this in wdioctl() for
ATAIOCCOMMAND.
 1.5 20-Nov-1998  thorpej Implement controller reference counting on ATA devices.
 1.4 20-Oct-1998  bouyer Donc reset the disk state to 0 in wd.c, set a flag which will cause it
to be reset to 0 at wdcstart(). This fixes a race condition between
normal I/O and wdioclt().
 1.3 13-Oct-1998  bouyer Add a new drive flag, DRIVE_MODE, set when the drive properly reported its
PIO/DMA modes. Don't try a SET_FEATURE if it didn't report its mode.
 1.2 12-Oct-1998  bouyer Merge bouyer-ide
 1.1 04-Jun-1998  bouyer branches: 1.1.2;
file atavar.h was initially added on branch bouyer-ide.
 1.1.2.5 04-Oct-1998  bouyer atavar.h: drv_softc is a struct device * instead of void*, as it's mostly
used for dv_xname
wd.c: convert for drv_softc type change, printf cleanup
wdc.c: always call ata_get_params() (params was used initialised with
non-32bit controllers, leaving to bogus PIO/DMA mode report).
Cleaup of the PIO/DMA mode message.
pciide_piix_reg.h: Fix definition of PIIX_IDETIM_CLEAR (unused before)
pciide.c: add a method do disable a channel on know device. If a channel
doesn't have any drive, we disable it and free its resources if disable
was successfull. This should help with laptops where the second channel
of the PIIX4 is unused but not disabled by BIOS. On such laptops,
irq15 can be used for PCMCIA but it was claimed by pciide.
Misc printf cleanup.
wdc_isa.c: printf cleanup.
 1.1.2.4 20-Sep-1998  bouyer Kill the 'old drive' flag. The probe for IDE controllers now looks
like this :
- if status = 0xff for both master and slave, no drive at all -> exit.
- issue a reset (timeout = 31s).
- test signatures: if ATAPI signature, flag a (possible) ATAPI devices here
else test register writability, then wait for the drive to become
ready, issue a diagnostic command and wait for the drive to become ready
if ok, flag a real ATA device.
This looks more like the old probe.
For ATA devices, if IDENTIFY fails, then it's an old drive and use
a fake disklabel/geometry.
 1.1.2.3 20-Sep-1998  bouyer - always call config_found() from wdcattach(), instead of printing our own
the "not configured" message ourselve. When no atapibus is configured,
use a fake ata_atapi_attach, else call wdc_atapibus_attach(). This way we
don't have to include the whole atapi support in an ata-only config.
- the dma_* function pointers take a 'int flags' as last argument, instead
of an 'int read'. The 2 valid flags are: WDC_DMA_READ (to select read/write)
and WDC_DMA_POLL, to signal interrupt-less mode (for core dumps).
- Reworked wdcprobe() so look more like the old one. A status of 0xff is
interpreted as "no drive" (freebsd does this); this this speed up
the probe for non-IDE machines (a reset timeout has been bumped to 31s,
to match the specs). The probe set ups the drive flags to either ATA or
ATAPI (depend on register signature) or OLD (if register signature don't
match, but a working drive appears to be there). Later the ATA code
will look at the OLD flag to decide wether there is no drive, or an old
(ST506) one if WDC_IDENTIFY fails.
- For known PCI chips, don't use the wdcprobe() heuristic to find wether
a channel is enabled/disabled; use the chip's specific registers for this.
- Clear the command queue when we have a polled command. This allows dumps to
work even if a command was being handled by the controller at panic time.
- Increase the number of available DMA segments by one, as user requests may
not be aligned on a page boundary.
 1.1.2.2 13-Aug-1998  bouyer - sync with HEAD
- better error detection and report for wd drives
- better wddump() support
- wdcwait() now takes a timout argument, so that we can honnor the
timeout argument of scsipi cmds.
 1.1.2.1 04-Jun-1998  bouyer Commit changes to the IDE system in a branch. This allows a better separation
between higth-level and low-level (i.e. registers read/write) and generalize
the queue for all commands. This also add supports for IDE DMA.
 1.13.12.2 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.13.12.1 21-Dec-1999  wrstuden Initial commit of recent changes to make DEV_BSIZE go away.

Runs on i386, needs work on other arch's. Main kernel routines should be
fine, but a number of the stand programs need help.

cd, fd, ccd, wd, and sd have been updated. sd has been tested with non-512
byte block devices. vnd, raidframe, and lfs need work.

Non 2**n block support is automatic for LKM's and conditional for kernels
on "options NON_PO2_BLOCKS".
 1.13.10.1 15-Nov-1999  fvdl Sync with -current
 1.13.8.3 27-Mar-2001  bouyer Kill scsipi_link from comment.
 1.13.8.2 20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
A i386 GENERIC kernel compiles without the siop, ahc and bha drivers
(will be updated later). i386 IDE/ATAPI and ncr work, as well as
sparc/esp_sbus. alpha should work as well (untested yet).
siop, ahc and bha will be updated once I've updated the branch to current
-current, as well as machine-dependant code.
 1.13.8.1 20-Oct-1999  thorpej Sync w/ trunk.
 1.13.2.2 07-Jul-2000  he Apply patch (requested by bouyer):
Add support for the following PCIIDE controllers:
o AMD 756
o CMD PCI0648 and PCI0649
o Hightpoint HPT366
o OPTi 82c621 (and a few of its derivatives)
o Promise Ultra/33 and Ultra/66
o Intel 82801 (ICH/ICH0)
Also fix PR#10437 (detect more ATAPI devices).
 1.13.2.1 23-Jan-2000  he Pull up revision 1.15 (requested by bouyer):
Improve the downgrade logic to work better in some cases (especially
for Acer Labs M5229 controllers with Ultra-DMA/66 disks).
 1.18.6.5 13-Aug-2002  nathanw Catch up to -current.
 1.18.6.4 20-Jun-2002  nathanw Catch up to -current.
 1.18.6.3 17-Apr-2002  nathanw Catch up to -current.
 1.18.6.2 08-Jan-2002  nathanw Catch up to -current.
 1.18.6.1 21-Jun-2001  nathanw Catch up to -current.
 1.18.4.1 23-Jan-2003  msaitoh Pull up revision 1.24 (requested by Manuel Bouyer):

read status after data commands in wdc_exec_command(). Mandatory for data out
commands (nonw are used for now), and doesn't hurt for data in. Also keep
track if we did transfer data. Makes the probe more robust and solves
kern/16159
 1.19.2.3 06-Sep-2002  jdolecek sync kqueue branch with HEAD
 1.19.2.2 23-Jun-2002  jdolecek catch up with -current on kqueue branch
 1.19.2.1 10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.25.4.1 12-May-2004  tron Pull up revision 1.41 via patch (requested by bouyer in ticket #1689):
Change the value of DRIVE_ATAPIST which was same as that of DRIVE_DMAERR.
Ok'd by bouyer.
 1.25.2.1 29-Aug-2002  gehenna catch up with -current.
 1.26.6.9 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.26.6.8 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.26.6.7 04-Feb-2005  skrll Sync with HEAD.
 1.26.6.6 02-Nov-2004  skrll Sync with HEAD.
 1.26.6.5 21-Sep-2004  skrll Fix the sync with head I botched.
 1.26.6.4 18-Sep-2004  skrll Sync with HEAD.
 1.26.6.3 25-Aug-2004  skrll Sync with HEAD.
 1.26.6.2 12-Aug-2004  skrll Sync with HEAD.
 1.26.6.1 03-Aug-2004  skrll Sync with HEAD
 1.40.2.2 11-Aug-2004  jmc branches: 1.40.2.2.2;
Pullup rev 1.44 (requested by bouyer in ticket #733)

Implement asynchronous channel reset.
Use this to reset the channel before doing a dump, instead of the hack in
wdc_exec_xfer() based on C_POLL. This hack was causing problems on
controllers with a shared queue, because we now can have C_POLL set during
concurent channels probes
 1.40.2.1 16-Apr-2004  tron Pull up revision 1.41 (requested by minoura in ticket #137):
Change the value of DRIVE_ATAPIST which was same as that of DRIVE_DMAERR.
Ok'd by bouyer.
 1.40.2.2.2.2 16-Apr-2005  tron Pull up revision 1.65 via patch (requested by bouyer in ticket #1105):
Allow polled commands to be queued, if ata_exec_xfer() is called in thread
context. For this introduce 3 new xfer tags:
- C_WAIT, equivalent of AT_WAIT
- C_WAITACT, when there is a thread waiting on this xfer to become the active
one (that is, to be at the head of the queue)
- C_FREE, set by ata_free_xfer() when it can't free the xfer because it's
still in use. The holder should then free the xfer ASAP.
If ata_exec_xfer() is called with (C_POLL | C_WAIT), and there is already
xfers to be processed, assert C_WAITACT and sleep.
atastart() checks for C_WAITACT, and wakeup the thread waiting for this xfer
to become active if set. atastart() won't process this xfer, it's the
responsability of the thread waked up to handle it.
Fix (the right way) kern/27421 by Martin Husemann.
 1.40.2.2.2.1 16-Mar-2005  jmc Pullup rev 1.43 (requested by mycroft in ticket #1018)

Only downgrade modes due to an actual CRC error. Downgrading on other errors
is anti-social -- especially given that there's no way to upgrade again short
of rebooting.

Also, downgrade UDMA modes more slowly. It's entirely possible that they're
using an 80-wire cable, but it's just too long for the higher modes, or there
is minor crosstalk.
 1.65.6.2 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.65.6.1 12-Feb-2005  yamt sync with head.
 1.65.4.1 29-Apr-2005  kent sync with -current
 1.67.2.1 27-May-2005  riz Pull up revision 1.68 (requested by bouyer in ticket #328):
Add a ata_queue_idle() function, which freeze a queue and tsleep() until the
controller is idle.
Change the powerhook function to call ata_queue_idle() on standby/suspend,
and ata_reset_channel() on resume (to wake up the disks and start from a
clean state).
Fix PR kern/30194 by Lubomir Sedlacik.
 1.69.2.5 24-Mar-2008  yamt sync with head.
 1.69.2.4 21-Jan-2008  yamt sync with head
 1.69.2.3 03-Sep-2007  yamt sync with head.
 1.69.2.2 30-Dec-2006  yamt sync with head.
 1.69.2.1 21-Jun-2006  yamt sync with head.
 1.71.8.1 14-Sep-2006  yamt sync with head.
 1.71.4.1 09-Sep-2006  rpaulo sync with head
 1.72.4.1 22-Oct-2006  yamt sync with head
 1.72.2.1 18-Nov-2006  ad Sync with head.
 1.73.10.1 11-Jul-2007  mjf Sync with head.
 1.73.8.1 09-Apr-2007  ad - Add two new arguments to kthread_create1: pri_t pri, bool mpsafe.
- Fork kthreads off proc0 as new LWPs, not new processes.
 1.74.18.1 11-Dec-2007  yamt sync with head.
 1.74.16.1 26-Dec-2007  ad Sync with head.
 1.74.8.2 23-Mar-2008  matt sync with HEAD
 1.74.8.1 09-Jan-2008  matt sync with HEAD
 1.74.6.2 06-Nov-2007  joerg Refactor PNP API:
- Make suspend/resume directly a device functionality. It consists of
three layers (class logic, device logic, bus logic), all of them being
optional. This replaces D0/D3 transitions.
- device_is_active returns true if the device was not disabled and was
not suspended (even partially), device_is_enabled returns true if the
device was enabled.
- Change pnp_global_transition into pnp_system_suspend and
pnp_system_resume. Before running any suspend/resume handlers, check
that all currently attached devices support power management and bail
out otherwise. The latter is not done for the shutdown/panic case.
- Make the former bus-specific generic network handlers a class handler.
- Make PNP message like volume up/down/toogle PNP events. Each device
can register what events they are interested in and whether the handler
should be global or not.
- Introduce device_active API for devices to mark themselve in use from
either the system or the device. Use this to implement the idle handling
for audio and input devices. This is intended to replace most ad-hoc
watchdogs as well.
- Fix somes situations in which audio resume would lose mixer settings.
- Make USB host controllers better deal with suspend in the light of
shared interrupts.
- Flush filesystem cache on suspend.
- Flush disk caches on suspend. Put ATA disks into standby on suspend as
well.
- Adopt drivers to use the new PNP API.
- Fix a critical bug in the generic cardbus layer that made D0->D3
break.
- Fix ral(4) to set if_stop.
- Convert cbb(4) to the new PNP API.
- Apply the PCI Express SCI fix on resume again.
 1.74.6.1 03-Aug-2007  jmcneill Pull in power management changes from private branch.
 1.75.10.3 17-Jan-2009  mjf Sync with HEAD.
 1.75.10.2 05-Oct-2008  mjf Sync with HEAD.
 1.75.10.1 03-Apr-2008  mjf Sync with HEAD.
 1.75.6.1 24-Mar-2008  keiichi sync with head.
 1.76.10.2 13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.76.10.1 19-Oct-2008  haad Sync with HEAD.
 1.76.6.1 10-Oct-2008  skrll Sync with HEAD.
 1.76.4.2 11-Mar-2010  yamt sync with head
 1.76.4.1 16-May-2009  yamt sync with head
 1.77.4.1 20-Nov-2008  snj Pull up following revision(s) (requested by bouyer in ticket #67):
sys/dev/ic/wdc.c: revision 1.256
sys/dev/ata/ata.c: revision 1.102
sys/dev/scsipi/atapi_wdc.c: revision 1.109
sys/dev/ata/atavar.h: revision 1.78
sys/dev/ata/ata_wdc.c: revision 1.91
cpu_intr_p() doesn't account for software interrupts (e.g. callouts) so
we can't use it here. Rssurect ATACH_TH_RUN, backing out
src/sys/dev/ata/ata.c 1.101
src/sys/dev/ata/ata_wdc.c 1.90
src/sys/dev/ata/atavar.h 1.77
src/sys/dev/ic/wdc.c 1.255
src/sys/dev/scsipi/atapi_wdc.c 1.108
Should fix kern/39927 and kern/39725.
 1.77.2.1 19-Jan-2009  skrll Sync with HEAD.
 1.78.4.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.80.6.1 06-Jun-2011  jruoho Sync with HEAD.
 1.80.4.2 31-May-2011  rmind sync with head
 1.80.4.1 21-Apr-2011  rmind sync with head
 1.83.6.1 18-Feb-2012  mrg merge to -current.
 1.83.2.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.83.2.2 30-Oct-2012  yamt sync with head
 1.83.2.1 17-Apr-2012  yamt sync with head
 1.90.2.2 03-Dec-2017  jdolecek update from HEAD
 1.90.2.1 23-Jun-2013  tls resync from head
 1.92.8.29 29-Sep-2017  jdolecek introduce ATA_BSIZE and use it instead of DEV_BSIZE for get params and
recovery, where they are by spec 512 bytes and not negotiable
 1.92.8.28 27-Sep-2017  jdolecek restore the atac_claim_hw and atac_free_hw hooks, they are used on atari
 1.92.8.27 26-Sep-2017  jdolecek make compile without NATA_DMA
 1.92.8.26 19-Sep-2017  jdolecek replace all remaining tsleep()/wakeup() calls with condition variables, or
calls to ata_delay(), as appropriate; change ata_delay() to require the
channel lock on entry, and pass the lock to kpause() for unlocking while
sleeping
 1.92.8.25 10-Sep-2017  jdolecek refactor code so that xfer c_start() hook is called with channel mutex held,
and hence the controller submit code no longer relies on spl

tested all the affected drivers - wdc (via piixide), ahci, mvsata, siisata,
both disk and atapi I/O
 1.92.8.24 12-Aug-2017  jdolecek do not reset drive after successful NCQ error recovery
 1.92.8.23 12-Aug-2017  jdolecek convert the atabus thread to use the channel lock and a condvar, adjust
code which sets the relevant channel flags to take the lock while doing so
 1.92.8.22 12-Aug-2017  jdolecek remove all logic around ATACH_IRQ_WAIT and channel-global ch_error/ch_status,
so that there is less hidden state shared by commands; primary intent is
to make the NCQ and non-NCQ paths more similar, and remove possibility
of incorrect handling for the NCQ commands

tested both disk and ATAPI - piixide(4) on QEMU, and siisata(4),
ahcisata(4), mvsata(4) on real hw
 1.92.8.21 01-Aug-2017  jdolecek move the drive recovery block to drive struct, it's inherently per-drive
 1.92.8.20 29-Jul-2017  jdolecek reserve the highest slot for error recovery, and also have ata_channel
include space for the READ LOG EXT sector, so that it's not necessary
to allocate memory on the error handling path; now ata_read_log_ext_ncq()
will never fail due to resource shortage
 1.92.8.19 22-Jul-2017  jdolecek provide channel flag when executing NCQ commands, so that e.g. intr
handler can use this for handling decisions without checking xfer
 1.92.8.18 21-Jul-2017  jdolecek KASSERT() that chaosed xfer actually ends up with error; might end up being
returned as successful due to bugs in error recovery code
 1.92.8.17 19-Jul-2017  jdolecek tighen and expand error handling, mostly for NCQ use cases:
- make retry timeout callout per xfer, i.e. retry separately
- zero whole bio struct on retry to avoid more stale state
- add a REQUEUE option, which doesn't bump retry count
- add ata_read_log_ext_ncq() for NCQ recovery
- adjust logic for activating xfers - allow next command only when
it's for same drive, several concurrent are only supported when HBA
and driver support FIS-based switching
- add new ata_timeout() which handles race between callout_stop()
and the invokation, add appropriate handling on deactivate/free paths
- stop using ch_status/ch_error in non-wdc code; later it will be dropped
completely
 1.92.8.16 27-Jun-2017  jdolecek attend error paths, more strict asserts and code consistency

- atastart() and ata_kill_pending() now KASSERT() that all xfers on queue
have same channel
- inactive xfers are killed via new reason KILL_GONE_INACTIVE, controller
code must not call any resource deactivation in that case
- c_intr() must call ata_waitdrain_xfer_check() as first thing, and must not
further touch any xfer structures on exit path; any resource cleanup
is supposed to be done in c_kill_xfer()
- c_kill_xfer() should never call atastart()
- ata_waitdrain_check() removed, replaced by ata_waitdrain_xfer_check()
- ATA_DRIVE_WAITDRAIN handling converted to use condvar
- removed unused ata_c callback
 1.92.8.15 23-Jun-2017  jdolecek restart I/O processing after freeing xfer, i.e. now even after commands
like cache flush or standby; the command handling no longer use on-stack xfer,
hence use queue slot and compete with normal I/O for the xfers

the restart give change to all drives attached to the same channel
in round-robin fashion, for fair usage and to recover from situation
when disk is idle due to all xfers being consumed by other drives

make special concession for flush cache - ignore any new I/O requests
on the particular disk while the flush cache is waiting for xfer,
so that I/O queue won't starve the flush cache and the flush cache would
be done ASAP

tested on piixide(4), ahci(4), siisata(4)
 1.92.8.14 21-Jun-2017  jdolecek hold channel lock for ata_exec_xfer() and most of atastart(), convert
C_WAITACT handling to use condvar

add comment for the code block with C_FREE and ata_free_xfer() explaining
why it's not abstraction layer violation
 1.92.8.13 21-Jun-2017  jdolecek change ata_queue_hwslot_to_xfer() and ata_queue_get_active_xfer() to take
ata_channel instead of ata_queue as parameter, and lock the channel while
traversing the queue
 1.92.8.12 21-Jun-2017  jdolecek actually remove the KASSERT() for queue_active in wdc.c as it's superfluous,
and make struct ata_queue completly opaque to attachment code; fix two other
places which were not converted to ata_queue_alloc() which were surfaced
by this change
 1.92.8.11 20-Jun-2017  jdolecek remove ata queue downsizing - every device, attached to the same channel,
uses slots according to it's own limits

wdc code changed to expect maximum one active xfer, and not check number
of openings in the channel; this is to facilitate using wdc functions
for e.g. handling of atapi commands for drivers which support both ATAPI
and NCQ
 1.92.8.10 19-Jun-2017  jdolecek add ata_channel lock, use it to protect queue manipulation (only that for now);
add ata_channel_detach() to destroy the locks

change ata_get_xfer() so that it can wait for xfer, convert all on-stack
xfer code to use the blocking variant

fix siisata_reset_drive() to use polled reset and not try ata_activate_xfer(),
convert drive probe code also over from slot0 XXX to ata_get_xfer()

drive reset and PMP now works on siisata(4) too; changes tested also
on piixide(4), ahci(4), mvsata(4)
 1.92.8.9 16-Jun-2017  jdolecek adjust reset channel and dump paths
- channel reset now always kills active transfer, even on dump path, but
now doesn't touch the queued waiting transfers; also kill_xfer hook is
always called, so that HBA can free any private xfer resources and thus
the dump request has chance to work
- kill_xfer routines now always call ata_deactivate_xfer(); added KASSERT()s
to ata_free_xfer() to expect deactivated xfer
- when called during channel reset before dump, ata_kill_active() drops
any queued waiting transfers without processing
- do not (re)queue any transfers in wddone() when dumping
- kill AT_RST_NOCMD flag

This should also hopefully fix the 'polled command has been queued' panic
as reported in:
PR kern/11811 by John Hawkinson
PR kern/47041 by Taylor R Campbell
PR kern/51979 by Martin Husemann

dump tested working with piixide(4) and ahci(4). mvsata(4) dump times out,
but otherwise tested working, will be fixed separately. siisata(4) mechanically
changed and not tested.
 1.92.8.8 24-Apr-2017  jdolecek set NCQ priority field to 'high' for BPRIO_TIMECRITICAL transfers if drive
supports it
 1.92.8.7 20-Apr-2017  jdolecek use a fake ata_channel struct in umass_isdata.c so that the wd(4) calls
to ata_{get,free}_xfer() can work, adjust to ata interface changes

compile tested only
 1.92.8.6 19-Apr-2017  jdolecek add ATA FUA support
 1.92.8.5 19-Apr-2017  jdolecek adjust ata code to support more than one active command, including the
timeout handling, add support for NCQ commands

move probe for NCQ and number of tags to middle layer, negotiate mutual
support between drive and controller

implement NCQ support in ahci(4)
 1.92.8.4 15-Apr-2017  jdolecek pass also ata_command via ata_xfer, callers of ata_exec_command() is now
responsible for allocation/disposal of the structure

change code to allocate ata_xfer for commands on stack same way as previously
the ata_command were, using c_slot 0; adjust asserts so that it would allow
several xfers with same c_slot, as long as only one such transfer is active
at a time
 1.92.8.3 15-Apr-2017  jdolecek make ata_xfer's allocated as part of ata_queue and make it include ata_bio;
they are pre-allocated on attach and ata_get_xfer() now never sleep, drop the
pool

modify wd(4) to file the bio requests using the xfers and hence
make it possible to have more than one active I/O request in flight;
ata_bio callback doesn't need to allocate any memory any more,
require it to never return ATACMD_TRY_AGAIN

move lp, badsect, multi from ata_bio to ata_drive_datas, as they are per-drive,
not per transfer

drop unused drv atac_claim_hw/atac_free_hw hooks, and also drop again ata_bio
c_hwslot
 1.92.8.2 11-Apr-2017  jdolecek hide active_xfer from drivers
 1.92.8.1 10-Apr-2017  jdolecek ATA infrastructure improvements to eventually support more outstanding
commands

patch by Matt Thomas
 1.95.2.4 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.95.2.3 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.95.2.2 25-Jun-2018  pgoyette Sync with HEAD
 1.95.2.1 22-Apr-2018  pgoyette Sync with HEAD
 1.97.2.3 21-Apr-2020  martin Sync with HEAD
 1.97.2.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.97.2.1 10-Jun-2019  christos Sync with HEAD
 1.99.2.12 15-Oct-2018  jdolecek change the SATA/NCQ recovery to run in the atabus thread
 1.99.2.11 11-Oct-2018  jdolecek refactor shared parts of the SATA error recovery into new function
ata_recovery_resume() and use for ahcisata/siisata/mvsata, also replace
per-controller hold/unhold with generic version

move the shared recovery code into separate file ata_recovery.c
 1.99.2.10 06-Oct-2018  jdolecek actually, just make dump use the same queue skip as recovery, and remove the
no longer necessary ata_queue_reset() call from wd(4)

also for PR kern/47041
 1.99.2.9 06-Oct-2018  jdolecek remove AT_RST_EMERG, do the queue reset explicitly in wd(4)

this should explicitly fix PR kern/47041 with sync during heavy
disk activity, even thought it was actually already implicitly fixed by calling
ata_thread_run() for drive reset in previous commit already, since the
function already called ata_queue_reset()

drop now unused ch_reset_flags and drive_reset_flags
 1.99.2.8 03-Oct-2018  jdolecek change channel reset and drive reset for all ATA controllers to always
run via thread, and with channel lock held the whole time; the queue is
frozen while reset is pending

for this repurpose ata_reset_channel() into new ata_thread_run()

also adjust some device printfs to not leak xfer pointer, and avoid
aprint_* for non-autoconf messages
 1.99.2.7 22-Sep-2018  jdolecek remove explicit ata_channel_start() calls, it's no longer necessary
now that ata_xfer's are allocated via pool and not really limited;
replace by just a callout to restart the processing for rare cases
where system runs out of memory
 1.99.2.6 22-Sep-2018  jdolecek fix use-after-free in wd(4) dump, detected by switch to the pool

change code in wd_dumpblocks() to use it's own non-pool ata_xfer,
which skips the deallocation step and thus keeps the contents when the I/O
is finished
 1.99.2.5 22-Sep-2018  jdolecek separate ata_xfer slot allocation and the memory allocation, so that
there can be more queued xfers than number of supported slots by controller,
and use a pool instead of custom pre-allocation

primarily to help PR kern/52614

remove no longer needed custom wd(4) logic for flush cache

switch also wd(4) trim/suspend/setcache/wdioctlstrategy to sleep waiting
for the memory, they are all called from process context and this
avoids spurious failures
 1.99.2.4 17-Sep-2018  jdolecek move ATAPI-only members of ata_xfer to an union struct to further save space
 1.99.2.3 17-Sep-2018  jdolecek switch from TAILQ to SIMPLEQ for ata_xfer pending queue to save
space, don't need doubly linked queue
 1.99.2.2 17-Sep-2018  jdolecek move low-level protocol handlers hooks from ata_xfer to separate struct,
initialized statically

primarily to reduce ata_xfer struct size, but also improves readibility,
and enforces consistency
 1.99.2.1 31-Aug-2018  jdolecek refactor ata_xfer to be just dumb structure; move all callouts/condvars out

retry callout to wd(4); reset callout and the active/cmd finish condvars
to channel queue; change code using the condvars so it works if there
are multiple waiters

simplify the async wait code for cmds, replace ata_wait_xfer()/ata_wake_xfer()
with ata_wait_cmd()

fix the callout_invoking/ack race handling code for timeouts to
actually have chance to work; change mvsata(4) to use generic timeout func

towards resolution of kern/52614
 1.103.4.1 30-Dec-2022  martin Pull up following revision(s) (requested by tsutsui in ticket #1557):

sys/dev/ic/ahcisata_core.c: revision 1.83
sys/dev/ic/ahcisata_core.c: revision 1.102
sys/dev/ata/ata.c: revision 1.164
sys/dev/ata/ata_wdc.c: revision 1.115
sys/dev/ata/ata_recovery.c: revision 1.4
sys/dev/ic/siisata.c: revision 1.42
sys/dev/ic/wdc.c: revision 1.308
sys/dev/ic/mvsata.c: revision 1.56
sys/dev/scsipi/atapi_wdc.c: revision 1.138
sys/dev/ic/siisata.c: revision 1.49
sys/dev/ata/atavar.h: revision 1.105
sys/dev/ata/wd.c: revision 1.460
sys/dev/ata/ata.c: revision 1.155
sys/dev/ata/wd.c: revision 1.462
sys/dev/ata/atavar.h: revision 1.109
sys/dev/ata/satapmp_subr.c: revision 1.16
sys/dev/ic/wdc.c: revision 1.299
sys/dev/ic/ahcisata_core.c: revision 1.93
sys/dev/ata/ata_wdc.c: revision 1.120
sys/dev/ic/wdcvar.h: revision 1.100
sys/dev/scsipi/atapi_wdc.c: revision 1.141
sys/dev/ic/mvsata.c: revision 1.61
sys/dev/usb/umass_isdata.c (apply patch)

drop wd lock in wdstart1() before calling the ata_bio hook; when called
from ata thread context, that can still need to sleep for wdc attachments
in wdcwait()

fix use-after-free for ata xfer on bio submission found by KASAN
driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself
PR kern/55169 by Nick Hudson

Function declaration formating whitespace consistency. NFCI.

PR kern/56403
Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.
(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.
Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().
(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).
"Go ahead" by jdolecek@.
 1.104.2.2 25-Apr-2020  bouyer Sync with bouyer-xenpvh-base2 (HEAD)
 1.104.2.1 20-Apr-2020  bouyer Sync with HEAD
 1.33 17-Feb-2025  jakllsch Restore ATA DMA mode downgrade support everywhere; it's a necessary part
of any system supporting (parallel) ATA DMA. There is hardware out there,
including cobalt, macppc, and sparc64 where this functionality is
necessary to avoid non-functional disks, either in as-shipped hardware
configurations or with add-in cards, or perhaps just with compromised
IDE/PATA cables.

Should address:
PR 58767
PR 59023
PR 59078

If anyone really insists on not having this support they can now turn it
off themselves with `options ATA_NO_DOWNGRADE_MODE`
 1.32 22-Sep-2024  uwe files.ata: defparam ATADEBUG_WD_MASK
 1.31 17-Jul-2023  riastradh branches: 1.31.6;
ata(4): Add ATA_DOWNGRADE_MODE to opt_ata.h.

This way adding it to kernel config will trigger recompilation.
 1.30 24-Oct-2018  jdolecek only include umass_isdata.c if atabus present, and likewise ata_subr.c
 1.29 24-Oct-2018  jdolecek only include ata_recovery.c with atabus (same as ata.c which provides
ata_thread_run() used here)
 1.28 22-Oct-2018  jdolecek Merge jdolecek-ncqfixes branch

- ata_xfer's are dynamicall allocated as needed using a pool, no longer
limited to number of possible openings supported by controller; dump
and recovery paths use dedicated pre-allocated storage
- moved callouts and condvars from ata_xfer to queue or channel, so that
ata_xfer does not need special initialization
- slot allocation now done when xfer is being activated, uncoupled
from memory allocation; active slots are no longer tracked by controller
code
- channel and drive reset is done always via the atabus thread, and
now executes with channel locked the whole time
- NCQ recovery moved to shared function, and run via the thread also
- added some workarounds for buggy error recovery AHCI emulation in QEMU
and Parallels

designed to primarily fix kern/52614, but might also help with kern/47041
and kern/53183
 1.27 10-Oct-2017  jdolecek branches: 1.27.2; 1.27.4; 1.27.6;
split off functions used by 'wd* at umass?' into separate file, unfortunately
the previous approach with NATABUS doesn't work for kernels which include
MODULAR, but not atabus - such as macppc and evbarm
 1.26 08-Oct-2017  jdolecek as a stopgap measure, include necessary parts of dev/ata/ata.c even
if no atabus is configured, so that kernels with just 'wd* at umass?'
and no atabus compile; fixes kernel builds for vax, pmax and evbmips
after NCQ branch merge

XXX wd* at umass? still actually untested after the merge, need to discuss
XXX what to do with it
 1.25 07-Oct-2017  jdolecek Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.24 31-Jul-2012  bouyer branches: 1.24.2; 1.24.28;
Apply back changes that were reverted on Jul 24 and Jul 26 (general ata/wdc
cleanup and SATA PMP support), now that I'm back to fix the fallouts.
 1.23 26-Jul-2012  jakllsch Revert, with intention of restoring in a less invasive way, the SATA Port
Multiplier code.

ok christos@
 1.22 02-Jul-2012  bouyer Add sata Port MultiPlier (PMP) support to the ata bus layer,
as described in
http://mail-index.netbsd.org/tech-kern/2012/06/23/msg013442.html
PMP support in integrated to the atabus layer.
struct ata_channel's ch_drive[] is not dynamically allocated, and ch_ndrive
(renamed to ch_ndrives) closely reflects the size of the ch_drive[] array.
Add helper functions atabus_alloc_drives() and atabus_free_drives()
to manage ch_drive[]/ch_ndrives.
Add wdc_maxdrives to struct wdc_softc so that bus front-end can specify
how much drive they really support (master/slave or single).
ata_reset_drive() callback gains a uint32_t *sigp argument which,
when not NULL, will contain the signature of the device being reset.
While there, some cosmetic changes:
- added a drive_type enum to ata_drive_datas, and stop encoding the
probed drive type in drive_flags (we were out of drive flags anyway).
- rename DRIVE_ATAPIST to DRIVE_ATAPIDSCW to better reflect what this
really is
- remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas
- factor out the interpretation of SATA signatures in sata_interpet_sig()

propagate these changes to the ATA HBA drivers, and add support for PMP
to ahcisata(4) and siisata(4).

Thanks to:
- Protocase (http://www.protocase.com/) which provided a system
with lots of controllers, SATA PMP and drive slots
- Conservation Genomics Laboratory, Department of Biology, New Mexico State
University for hosting the above system
- Brook Milligan, who set up remote access and has been very responsive
when SATA cable move was needed
 1.21 17-Jun-2009  jakllsch branches: 1.21.12;
ahcisata(4) and siisata(4) use similar SATA FIS functions, share them.
 1.20 16-Sep-2008  tron branches: 1.20.4; 1.20.8; 1.20.12;
Support multiple volumes connected to Intel MatrixRAID controllers.
Code contributed by Juan RP in PR kern/39552.
 1.19 11-Sep-2008  tron Add support for Intel MatrixRAID to ataraid(4). It is currently limited
to one volume.

The driver was provided by Juan RP in PR kern/39511.
 1.18 05-Sep-2008  tron Add JMicron RAID support to ataraid(4).
Patch provided by Juan RP in PR kern/39395.
 1.17 20-Aug-2008  tacha Add support for nVidia MediaShield ata raid found on HP ProLiant ML115.
Tested on HEAD and netbsd-4 branch with RAID0, RAID1, SPAN with MCP55 SATA controller.
 1.16 10-Jun-2008  drochner branches: 1.16.2;
use new "defpseudodev" where appropriate, and remove dummy interface
attribute definitions which were only to trick config(8)
 1.15 27-Mar-2007  garbled branches: 1.15.30; 1.15.32; 1.15.34; 1.15.36; 1.15.38;
Add support for VIA V-TECH ata raid. Tested on 4.0 with RAID0, RAID1 and
SPAN over a pair of sata drives on a VT8237A SATA Controller.
 1.14 25-Oct-2006  bouyer branches: 1.14.2; 1.14.4; 1.14.8; 1.14.10; 1.14.12;
Exclude wdc_sataprobe() when no SATA controllers are present. Fix
build of *_TINY kernels.
 1.13 11-Dec-2005  christos branches: 1.13.20; 1.13.22;
merge ktrace-lwp.
 1.12 20-Jun-2005  briggs branches: 1.12.2;
Basic support for RAID0 and RAID1 for the Adaptec HostRAID format as found
on the Intel 6300ESB on-board RAID. Adapted from FreeBSD.
 1.11 30-Aug-2004  drochner branches: 1.11.10;
make the software managed bad-sector list optional
(kernel option WD_SOFTBADSECT)
 1.10 12-Aug-2004  thorpej Don't assume wdc-specific ATA / ATAPI code should be brought in if atabus
or atapibus are configured; use a separate wdc_common attribute to indicate
that the shared wdc code is also present.
 1.9 28-May-2004  thorpej Add common routines for SATA controllers. Right now, we have sata_speed(),
which reports the speed encoded in the SStatus register.
 1.8 08-Oct-2003  bouyer Following Matt Thomas's request, rename ata attribute to ata_hl, and
wdc_base to ata. We can now have
atabus* at ata?
in kernel config files.
 1.7 08-Oct-2003  bouyer Make the ATA mid-layer appears as atabus, as proposed in
http://mail-index.netbsd.org/tech-kern/2003/09/25/0006.html
This adds a device (atabus) between IDE controllers and wd or atapibus, to
have each ATA channel show up in the device tree. Later there will be atabus
devices in /dev, so that we can do IOCTL on them.
Each atabus has its own kernel thread, to handle operations that needs polling,
e.g. reset and others.

Device probing on each bus it defered to the atabus thread creation.
This allows to do the reset and basic device probes in parallel, which reduce
boot time on systems with several pciide controllers.
 1.6 27-Jan-2003  thorpej branches: 1.6.2;
Experimental support for RAID volumes configured by ATA "RAID" controllers.
Such RAID controllers are actually just IDE controllers with a BIOS that
can create RAID volumes and write the configuration info to config blocks
on the disks. The BIOS can do I/O to these volumes, and the OS must
understand the config blocks and implement RAID in software in order to be
able to use these volumes.

Only SPAN (simple concatenation) and RAID0 are supported at this time,
and writing back config blocks is also not supported at this time. Currently,
only the Promise configuration scheme is supported, although supporting
the Highpoint scheme should not be too difficult.

In any case, this is sufficient to use the Promise RAID0 volume (thus
preserving the win2k AS installation) on this new Intel server I have.

Thanks to Soren Schmidt for doing the work in FreeBSD; it made this
task much easier. The config block parsing code is adapted from his
work.
 1.5 24-Feb-2001  lukem whitespace police
 1.4 10-Apr-2000  bouyer Add dependancy to wdc for ata.c, so that 'atapibus at umass' will compile
without wdc or pciide. Closes kern/9842 from Gary Duzan.
 1.3 12-Oct-1998  bouyer branches: 1.3.12;
Merge bouyer-ide
 1.2 16-Feb-1998  thorpej branches: 1.2.2;
Remove "class" declarations, and add "devclass" declarations where
appropriate. Fix several inconsistencies between device class and
attributes. Mostly from Chris Demetriou.
 1.1 14-Jan-1998  cgd Shuffle wd/wdc driver files around (via repository copies). wdc
now lives in dev/ic, wd now lives in dev/ata. there's now a 'ata'
interface attribute defined in conf/files, but wdc can't go there
yet because some ports still use private versions based on the old
ISA version.
 1.2.2.2 15-Jun-1998  bouyer include ata_wdc.c only if wd is defined.
 1.2.2.1 04-Jun-1998  bouyer Commit changes to the IDE system in a branch. This allows a better separation
between higth-level and low-level (i.e. registers read/write) and generalize
the queue for all commands. This also add supports for IDE DMA.
 1.3.12.2 12-Mar-2001  bouyer Sync with HEAD.
 1.3.12.1 20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
A i386 GENERIC kernel compiles without the siop, ahc and bha drivers
(will be updated later). i386 IDE/ATAPI and ncr work, as well as
sparc/esp_sbus. alpha should work as well (untested yet).
siop, ahc and bha will be updated once I've updated the branch to current
-current, as well as machine-dependant code.
 1.6.2.6 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.6.2.5 21-Sep-2004  skrll Fix the sync with head I botched.
 1.6.2.4 18-Sep-2004  skrll Sync with HEAD.
 1.6.2.3 03-Sep-2004  skrll Sync with HEAD
 1.6.2.2 12-Aug-2004  skrll Sync with HEAD.
 1.6.2.1 03-Aug-2004  skrll Sync with HEAD
 1.11.10.1 03-Jul-2005  tron Pull up revision 1.12 (requested by briggs in ticket #521):
Basic support for RAID0 and RAID1 for the Adaptec HostRAID format as found
on the Intel 6300ESB on-board RAID. Adapted from FreeBSD.
 1.12.2.2 03-Sep-2007  yamt sync with head.
 1.12.2.1 30-Dec-2006  yamt sync with head.
 1.13.22.1 10-Dec-2006  yamt sync with head.
 1.13.20.1 18-Nov-2006  ad Sync with head.
 1.14.12.1 29-Mar-2007  reinoud Pullup to -current
 1.14.10.1 11-Jul-2007  mjf Sync with head.
 1.14.8.1 10-Apr-2007  ad Sync with head.
 1.14.4.1 15-Apr-2007  yamt sync with head.
 1.14.2.1 31-Mar-2007  bouyer Pull up following revision(s) (requested by garbled in ticket #542):
sys/dev/ata/ata_raidreg.h: revision 1.4 via patch
sys/dev/ata/ata_raidvar.h: revision 1.5via patch
sys/dev/ata/files.ata: revision 1.15via patch
sys/dev/ata/ata_raid_via.c: revision 1.1
sys/dev/ata/ata_raid.c: revision 1.22via patch
Add support for VIA V-TECH ata raid. Tested on 4.0 with RAID0, RAID1 and
SPAN over a pair of sata drives on a VT8237A SATA Controller.
 1.15.38.1 18-Jun-2008  simonb Sync with head.
 1.15.36.3 24-Sep-2008  wrstuden Merge in changes between wrstuden-revivesa-base-2 and
wrstuden-revivesa-base-3.
 1.15.36.2 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.15.36.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.15.34.2 20-Jun-2009  yamt sync with head
 1.15.34.1 04-May-2009  yamt sync with head.
 1.15.32.1 17-Jun-2008  yamt sync with head.
 1.15.30.2 28-Sep-2008  mjf Sync with HEAD.
 1.15.30.1 29-Jun-2008  mjf Sync with HEAD.
 1.16.2.1 19-Oct-2008  haad Sync with HEAD.
 1.20.12.1 21-Apr-2010  matt sync to netbsd-5
 1.20.8.1 23-Jul-2009  jym Sync with HEAD.
 1.20.4.1 28-Sep-2009  snj Pull up following revision(s) (requested by jakllsch in ticket #1023):
sys/conf/files: revision 1.947
sys/dev/ata/files.ata: revision 1.21
sys/dev/ata/satafis_subr.c: revision 1.1
sys/dev/ata/satafisreg.h: revision 1.1
sys/dev/ata/satafisvar.h: revision 1.1
ahcisata(4) and siisata(4) use similar SATA FIS functions, share them.
 1.21.12.1 30-Oct-2012  yamt sync with head
 1.24.28.1 19-Jul-2017  jdolecek defflag WD_CHAOS_MONKEY into opt_wd.h together with WD_SOFTBADSECT to set/unset
this more easily
 1.24.2.1 03-Dec-2017  jdolecek update from HEAD
 1.27.6.1 11-Oct-2018  jdolecek refactor shared parts of the SATA error recovery into new function
ata_recovery_resume() and use for ahcisata/siisata/mvsata, also replace
per-controller hold/unhold with generic version

move the shared recovery code into separate file ata_recovery.c
 1.27.4.1 10-Jun-2019  christos Sync with HEAD
 1.27.2.1 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.31.6.1 02-Aug-2025  perseant Sync with HEAD
 1.52 13-Apr-2025  rin ld(4): Convert blkno argument for sc_dump() to daddr_t

PR kern/59153

(1) For backends that accept 64-bit block address, i.e.,
nvme(4), virtio(4), aac(4), iop(4), and mainbus(usermode/4),
this should enable to dump beyond 2Gi blocks.

(2) sdmmc(4) backend allows to dump up to the last block.

(3) For other backends, block address is handled as `int`.
Some of them may support blocks up to 4Gi, but I do not have
enough time to examine datasheets. So, continue to reject >2Gi
blocks as before.

XXX
This is KABI change, and cannot be pulled up into netbsd-{10,9}.

XXX
Compile-test only (for amd64/ALL) due to lack of large SSDs ;)

Thanks mlelstv@ for discussion and careful review!!
 1.51 16-Mar-2025  andvar ataraid(4), ld(4): ensure RAID disk status is set before resolving vnode.

Given a RAID array disk is missing (due to removal, failure, or a code bug),
its configuration is skipped. This causes ld(4) attachment at ataraid(4) to
fail because adi->adi_dev is NULL.

The fix will treat disks without known status (ADI_S_ONLINE, ADI_S_SPARE,
ADI_S_ASSIGNED) as if ata_raid_disk_vnode_find() returned NULL, more gracefully
solving above situation and avoiding the crash.

Additionally, replace vnode block device opening logic with vn_bdev_open()
(vn_subr(9)). This function provides identical functionality while also
incrementing vp->v_writecount, preventing a 'vp->v_writecount > 0` assertion
failure on vn_close().

More details are available in the tech-kern mailing list thread:
the https://mail-index.netbsd.org/tech-kern/2025/03/10/msg030179.html thread
and PR thread.

Should fix PR kern/43986
Also observed in PR kern/59130 (due to a different bug).

pullups are needed to netbsd-9, netbsd-10.
 1.50 17-Jan-2020  ad branches: 1.50.24; 1.50.30;
Acquire kernel_lock in the bp->b_iodone callback.
 1.49 27-Jan-2019  pgoyette branches: 1.49.6;
Merge the [pgoyette-compat] branch
 1.48 23-Oct-2018  jdolecek add support for DIOCCACHESYNC (!), and DIOCGCACHE; code adapted from ccd(4)

tested with Intel ATA Raid
 1.47 22-Oct-2018  jdolecek no need for <sys/malloc.h> here
 1.46 03-Sep-2018  riastradh Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)
 1.45 03-Jun-2018  martin branches: 1.45.2; 1.45.4;
Whitespace
 1.44 27-Sep-2016  pgoyette branches: 1.44.14;
Remove some debug printf's that slipped through. Thanks, wiz!
 1.43 27-Sep-2016  pgoyette Modularize the ld driver and all of its attachments. Ensure that all
parents are capable of rescan (or otherwise provide a means of attaching
children post-initialization).
 1.42 16-Sep-2016  jdolecek modify ldattach() to have default strategy as a parameter
 1.41 02-May-2016  christos branches: 1.41.2;
move scsipi_strvis -> libkern:strnvisx()
change the prototype to match userland
fix sizes of strings passed to it
 1.40 13-Apr-2015  riastradh Convert sys/dev to use <sys/rndsource.h>.
 1.39 02-Feb-2012  tls branches: 1.39.6; 1.39.24;
Entropy-pool implementation move and cleanup.

1) Move core entropy-pool code and source/sink/sample management code
to sys/kern from sys/dev.

2) Remove use of NRND as test for presence of entropy-pool code throughout
source tree.

3) Remove use of RND_ENABLED in device drivers as microoptimization to
avoid expensive operations on disabled entropy sources; make the
rnd_add calls do this directly so all callers benefit.

4) Fix bug in recent rnd_add_data()/rnd_add_uint32() changes that might
have lead to slight entropy overestimation for some sources.

5) Add new source types for environmental sensors, power sensors, VM
system events, and skew between clocks, with a sample implementation
for each.

ok releng to go in before the branch due to the difficulty of later
pullup (widespread #ifdef removal and moved files). Tested with release
builds on amd64 and evbarm and live testing on amd64.
 1.38 12-Jun-2011  rmind branches: 1.38.2; 1.38.6;
Welcome to 5.99.53! Merge rmind-uvmplock branch:

- Reorganize locking in UVM and provide extra serialisation for pmap(9).
New lock order: [vmpage-owner-lock] -> pmap-lock.

- Simplify locking in some pmap(9) modules by removing P->V locking.

- Use lock object on vmobjlock (and thus vnode_t::v_interlock) to share
the locks amongst UVM objects where necessary (tmpfs, layerfs, unionfs).

- Rewrite and optimise x86 TLB shootdown code, make it simpler and cleaner.
Add TLBSTATS option for x86 to collect statistics about TLB shootdowns.

- Unify /dev/mem et al in MI code and provide required locking (removes
kernel-lock on some ports). Also, avoid cache-aliasing issues.

Thanks to Andrew Doran and Joerg Sonnenberger, as their initial patches
formed the core changes of this branch.
 1.37 06-Jul-2010  bsh branches: 1.37.6;
* use correct interleave factor for RAID-0 of Intel MatrixRaid.
* report stripe size in Kibytes to bioctl(8).

Unfortunately this change will break compatibility with existing
filesystem made by NetBSD on RAID-0 Intel MatrixRAID volumes.

Please read: http://mail-index.netbsd.org/current-users/2010/07/02/msg013810.html
 1.36 06-Jul-2010  bsh fix LOCKDEBUG panic at ataraid(4).
the patch was provided by Juan RP in response to PR kern 38273.
(http://mail-index.netbsd.org/tech-kern/2008/09/17/msg002734.html)
 1.35 12-May-2009  cegger branches: 1.35.2; 1.35.4;
struct device * -> device_t, no functional changes intended.
 1.34 12-May-2009  cegger struct cfdata * -> cfdata_t, no functional changes intended.
 1.33 15-Oct-2008  wrstuden branches: 1.33.8;
Merge wrstuden-revivesa into HEAD.
 1.32 16-Sep-2008  tron Support multiple volumes connected to Intel MatrixRAID controllers.
Code contributed by Juan RP in PR kern/39552.
 1.31 15-Sep-2008  tron Only report volume names if they are non empty.
Patch provided by Juan RP in PR kern/39514.
 1.30 15-Sep-2008  tron Add support for status reports via bio(4) to ataraid(4).
The code was contributed by Juan RP in PR kern/39514.
 1.29 09-Sep-2008  tron Complete device_t/softc split for the ld(4) attachments. This should
prevent crashes while attaching a drive.

Patch supplied by Juan RP in PR kern/39468.
 1.28 24-Aug-2008  hannken Destroy cb_buf before pool_put().

Resolves PR #38636
 1.27 04-May-2008  xtraeme branches: 1.27.2; 1.27.6;
device_t/softc split.
 1.26 05-Apr-2008  cegger branches: 1.26.2; 1.26.4;
use aprint_*_dev and device_xname
 1.25 21-Mar-2008  ad Catch up with descriptor handling changes. See kern_descrip.c revision
1.173 for details.
 1.24 02-Jan-2008  ad branches: 1.24.6;
Another driver that should use getiobuf().
 1.23 02-Jan-2008  ad Merge vmlocking2 to head.
 1.22 26-Nov-2007  pooka branches: 1.22.2; 1.22.6;
Remove the "struct lwp *" argument from all VFS and VOP interfaces.
The general trend is to remove it from all kernel interfaces and
this is a start. In case the calling lwp is desired, curlwp should
be used.

quick consensus on tech-kern
 1.21 29-Jul-2007  ad branches: 1.21.4; 1.21.6; 1.21.12; 1.21.14;
It's not a good idea for device drivers to modify b_flags, as they don't
need to understand the locking around that field. Instead of setting
B_ERROR, set b_error instead. b_error is 'owned' by whoever completes
the I/O request.
 1.20 12-Mar-2007  ad branches: 1.20.8;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.
 1.19 04-Mar-2007  christos branches: 1.19.2;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
 1.18 16-Nov-2006  christos branches: 1.18.4;
__unused removal on arguments; approved by core.
 1.17 12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.16 14-May-2006  elad branches: 1.16.8; 1.16.10;
integrate kauth.
 1.15 28-Mar-2006  thorpej Use device_unit().
 1.14 11-Dec-2005  christos branches: 1.14.4; 1.14.6; 1.14.8; 1.14.10; 1.14.12;
merge ktrace-lwp.
 1.13 28-Oct-2004  yamt branches: 1.13.12;
move buffer queue related stuffs from buf.h to their own header, bufq.h.
 1.12 17-Sep-2004  enami Minimal support for mirror component. The relation between original and
mirror component learned from FreeBSD driver.
 1.11 22-Apr-2004  itojun sprintf -> snprintf
 1.10 25-Jan-2004  hannken Make VOP_STRATEGY(bp) a real VOP as discussed on tech-kern.

VOP_STRATEGY(bp) is replaced by one of two new functions:

- VOP_STRATEGY(vp, bp) Call the strategy routine of vp for bp.
- DEV_STRATEGY(bp) Call the d_strategy routine of bp->b_dev for bp.

DEV_STRATEGY(bp) is used only for block-to-block device situations.
 1.9 14-Dec-2003  thorpej Cosmetic changes in some autoconfiguration messages.
 1.8 14-Jul-2003  lukem add missing __KERNEL_RCSID()
 1.7 29-Jun-2003  fvdl branches: 1.7.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.
 1.6 28-Jun-2003  darrenr Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V
 1.5 07-Jun-2003  thorpej Always queue transfers in the ld driver to prevent transient resource
shortages in the back-ends from aborting transfers.

Reviewed by ad@.
 1.4 25-Feb-2003  thorpej Add a new BUF_INIT() macro which initializes b_dep and b_interlock, and
use it. This fixes a few places where either b_dep or b_interlock were
not properly initialized.
 1.3 29-Jan-2003  thorpej Don't set LDF_ENABLED unless the device is really okay. Noted by
Andrew Doran.
 1.2 27-Jan-2003  thorpej G/C something left over from an earlier version of this code.
 1.1 27-Jan-2003  thorpej Experimental support for RAID volumes configured by ATA "RAID" controllers.
Such RAID controllers are actually just IDE controllers with a BIOS that
can create RAID volumes and write the configuration info to config blocks
on the disks. The BIOS can do I/O to these volumes, and the OS must
understand the config blocks and implement RAID in software in order to be
able to use these volumes.

Only SPAN (simple concatenation) and RAID0 are supported at this time,
and writing back config blocks is also not supported at this time. Currently,
only the Promise configuration scheme is supported, although supporting
the Highpoint scheme should not be too difficult.

In any case, this is sufficient to use the Promise RAID0 volume (thus
preserving the win2k AS installation) on this new Intel server I have.

Thanks to Soren Schmidt for doing the work in FreeBSD; it made this
task much easier. The config block parsing code is adapted from his
work.
 1.7.2.5 02-Nov-2004  skrll Sync with HEAD.
 1.7.2.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.7.2.3 18-Sep-2004  skrll Sync with HEAD.
 1.7.2.2 03-Aug-2004  skrll Sync with HEAD
 1.7.2.1 02-Jul-2003  darrenr Apply the aborted ktrace-lwp changes to a specific branch. This is just for
others to review, I'm concerned that patch fuziness may have resulted in some
errant code being generated but I'll look at that later by comparing the diff
from the base to the branch with the file I attempt to apply to it. This will,
at the very least, put the changes in a better context for others to review
them and attempt to tinker with removing passing of 'struct lwp' through
the kernel.
 1.13.12.6 24-Mar-2008  yamt sync with head.
 1.13.12.5 21-Jan-2008  yamt sync with head
 1.13.12.4 07-Dec-2007  yamt sync with head
 1.13.12.3 03-Sep-2007  yamt sync with head.
 1.13.12.2 30-Dec-2006  yamt sync with head.
 1.13.12.1 21-Jun-2006  yamt sync with head.
 1.14.12.2 24-May-2006  tron Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.
 1.14.12.1 31-Mar-2006  tron Merge 2006-03-31 NetBSD-current into the "peter-altq" branch.
 1.14.10.2 06-May-2006  christos - Move kauth_cred_t declaration to <sys/types.h>
- Cleanup struct ucred; forward declarations that are unused.
- Don't include <sys/kauth.h> in any header, but include it in the c files
that need it.

Approved by core.
 1.14.10.1 19-Apr-2006  elad sync with head.
 1.14.8.2 24-May-2006  yamt sync with head.
 1.14.8.1 01-Apr-2006  yamt sync with head.
 1.14.6.2 01-Jun-2006  kardel Sync with head.
 1.14.6.1 22-Apr-2006  simonb Sync with head.
 1.14.4.1 09-Sep-2006  rpaulo sync with head
 1.16.10.2 10-Dec-2006  yamt sync with head.
 1.16.10.1 22-Oct-2006  yamt sync with head
 1.16.8.1 18-Nov-2006  ad Sync with head.
 1.18.4.2 24-Mar-2007  yamt sync with head.
 1.18.4.1 12-Mar-2007  rmind Sync with HEAD.
 1.19.2.6 09-Oct-2007  ad Sync with head.
 1.19.2.5 24-Aug-2007  ad Sync with buffer cache locking changes. See buf.h/vfs_bio.c for details.
Some minor portions are incomplete and needs to be verified as a whole.
 1.19.2.4 19-Aug-2007  ad - Back out the biodone() changes.
- Eliminate B_ERROR (from HEAD).
 1.19.2.3 17-Jun-2007  ad - Increase the number of thread priorities from 128 to 256. How the space
is set up is to be revisited.
- Implement soft interrupts as kernel threads. A generic implementation
is provided, with hooks for fast-path MD code that can run the interrupt
threads over the top of other threads executing in the kernel.
- Split vnode::v_flag into three fields, depending on how the flag is
locked (by the interlock, by the vnode lock, by the file system).
- Miscellaneous locking fixes and improvements.
 1.19.2.2 13-May-2007  ad - Pass the error number and residual count to biodone(), and let it handle
setting error indicators. Prepare to eliminate B_ERROR.
- Add a flag argument to brelse() to be set into the buf's flags, instead
of doing it directly. Typically used to set B_INVAL.
- Add a "struct cpu_info *" argument to kthread_create(), to be used to
create bound threads. Change "bool mpsafe" to "int flags".
- Allow exit of LWPs in the IDL state when (l != curlwp).
- More locking fixes & conversion to the new API.
 1.19.2.1 13-Mar-2007  ad Sync with head.
 1.20.8.1 15-Aug-2007  skrll Sync with HEAD.
 1.21.14.2 29-Jul-2007  ad It's not a good idea for device drivers to modify b_flags, as they don't
need to understand the locking around that field. Instead of setting
B_ERROR, set b_error instead. b_error is 'owned' by whoever completes
the I/O request.
 1.21.14.1 29-Jul-2007  ad file ld_ataraid.c was added on branch matt-mips64 on 2007-07-29 12:50:20 +0000
 1.21.12.2 18-Feb-2008  mjf Sync with HEAD.
 1.21.12.1 08-Dec-2007  mjf Sync with HEAD.
 1.21.6.1 09-Jan-2008  matt sync with HEAD
 1.21.4.1 27-Nov-2007  joerg Sync with HEAD. amd64 Xen support needs testing.
 1.22.6.2 08-Jan-2008  bouyer Sync with HEAD
 1.22.6.1 02-Jan-2008  bouyer Sync with HEAD
 1.22.2.1 04-Dec-2007  ad Pull the vmlocking changes into a new branch.
 1.24.6.4 17-Jan-2009  mjf Sync with HEAD.
 1.24.6.3 28-Sep-2008  mjf Sync with HEAD.
 1.24.6.2 02-Jun-2008  mjf Sync with HEAD.
 1.24.6.1 03-Apr-2008  mjf Sync with HEAD.
 1.26.4.4 11-Aug-2010  yamt sync with head.
 1.26.4.3 16-May-2009  yamt sync with head
 1.26.4.2 04-May-2009  yamt sync with head.
 1.26.4.1 16-May-2008  yamt sync with head.
 1.26.2.1 18-May-2008  yamt sync with head.
 1.27.6.1 19-Oct-2008  haad Sync with HEAD.
 1.27.2.2 24-Sep-2008  wrstuden Merge in changes between wrstuden-revivesa-base-2 and
wrstuden-revivesa-base-3.
 1.27.2.1 18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.33.8.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.35.4.2 05-Mar-2011  rmind sync with head
 1.35.4.1 16-Mar-2010  rmind Change struct uvm_object::vmobjlock to be dynamically allocated with
mutex_obj_alloc(). It allows us to share the locks among UVM objects.
 1.35.2.1 17-Aug-2010  uebayasi Sync with HEAD.
 1.37.6.1 23-Jun-2011  cherry Catchup with rmind-uvmplock merge.
 1.38.6.1 18-Feb-2012  mrg merge to -current.
 1.38.2.1 17-Apr-2012  yamt sync with head
 1.39.24.3 05-Oct-2016  skrll Sync with HEAD
 1.39.24.2 29-May-2016  skrll Sync with HEAD
 1.39.24.1 06-Jun-2015  skrll Sync with HEAD
 1.39.6.1 03-Dec-2017  jdolecek update from HEAD
 1.41.2.1 04-Nov-2016  pgoyette Sync with HEAD
 1.44.14.3 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.44.14.2 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.44.14.1 25-Jun-2018  pgoyette Sync with HEAD
 1.45.4.1 17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.45.2.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.45.2.1 10-Jun-2019  christos Sync with HEAD
 1.49.6.1 17-Jan-2020  ad Sync with head.
 1.50.30.1 02-Aug-2025  perseant Sync with HEAD
 1.50.24.1 07-May-2025  martin Pull up following revision(s) (requested by andvar in ticket #1100):

sys/dev/ata/ld_ataraid.c: revision 1.51
sys/dev/ata/ata_raid_subr.c: revision 1.5

ataraid(4), ld(4): ensure RAID disk status is set before resolving vnode.

Given a RAID array disk is missing (due to removal, failure, or a code bug),
its configuration is skipped. This causes ld(4) attachment at ataraid(4) to
fail because adi->adi_dev is NULL.

The fix will treat disks without known status (ADI_S_ONLINE, ADI_S_SPARE,
ADI_S_ASSIGNED) as if ata_raid_disk_vnode_find() returned NULL, more gracefully
solving above situation and avoiding the crash.

Additionally, replace vnode block device opening logic with vn_bdev_open()
(vn_subr(9)). This function provides identical functionality while also
incrementing vp->v_writecount, preventing a 'vp->v_writecount > 0` assertion
failure on vn_close().

More details are available in the tech-kern mailing list thread:
the https://mail-index.netbsd.org/tech-kern/2025/03/10/msg030179.html thread
and PR thread.
Should fix PR kern/43986

Also observed in PR kern/59130 (due to a different bug).
 1.24 21-Jun-2018  jdolecek split the port status reporting to new function sata_interpret_det()
so it can be called separately from sata_reset_interface()

do not treat PHY offline as an error, it's pretty normal when there
is no device actually connected

debugging aid for PR kern/52372
 1.23 07-Oct-2017  jdolecek branches: 1.23.2;
Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.22 10-May-2017  msaitoh branches: 1.22.2;
KNF. No functional change.
 1.21 03-Apr-2013  bouyer branches: 1.21.12; 1.21.24; 1.21.26;
Fix kernel dump on ahci controller, by making sure we won't sleep
while dumping:
- introduce ata_delay() which either use delay() or kpause()
depending on flags. use it in sata_reset_interface() and
some ahci functions
- kill ATA_NOSLEEP, it was tested but never set. use ATA_POLL instead.
- reduce delay while polling in ahci, to speed up the dump

Should fix PR kern/41095
 1.20 31-Jul-2012  bouyer branches: 1.20.2;
Apply back changes that were reverted on Jul 24 and Jul 26 (general ata/wdc
cleanup and SATA PMP support), now that I'm back to fix the fallouts.
 1.19 26-Jul-2012  jakllsch Revert, with intention of restoring in a less invasive way, the SATA Port
Multiplier code.

ok christos@
 1.18 24-Jul-2012  jakllsch Revert dsl@'s changes of Sun, 15 Jul 2012 10:55:35 +0000 and
Sun, 15 Jul 2012 10:56:50 +0000, excepting the kernel version bump.
First step in reverting regressions to ata(4) subsystem during the addition of
port multiplier support.
 1.17 15-Jul-2012  dsl Some namespace protection (and add greppablity).
Prefix the DRIVE_ and DRIVET_ constants from atavar.h with ATA_.
Don't use an enum for drive_type - you don't know how big it will be.
Move driver_type to avoid implicit structure padding (esp on arm).
This change is purely lexical and mechanical.

Update to 6.99.9 - this wasn't done when the SATA PMP changes
were made - I'm sure they warranted a bump.
 1.16 02-Jul-2012  bouyer Add sata Port MultiPlier (PMP) support to the ata bus layer,
as described in
http://mail-index.netbsd.org/tech-kern/2012/06/23/msg013442.html
PMP support in integrated to the atabus layer.
struct ata_channel's ch_drive[] is not dynamically allocated, and ch_ndrive
(renamed to ch_ndrives) closely reflects the size of the ch_drive[] array.
Add helper functions atabus_alloc_drives() and atabus_free_drives()
to manage ch_drive[]/ch_ndrives.
Add wdc_maxdrives to struct wdc_softc so that bus front-end can specify
how much drive they really support (master/slave or single).
ata_reset_drive() callback gains a uint32_t *sigp argument which,
when not NULL, will contain the signature of the device being reset.
While there, some cosmetic changes:
- added a drive_type enum to ata_drive_datas, and stop encoding the
probed drive type in drive_flags (we were out of drive flags anyway).
- rename DRIVE_ATAPIST to DRIVE_ATAPIDSCW to better reflect what this
really is
- remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas
- factor out the interpretation of SATA signatures in sata_interpet_sig()

propagate these changes to the ATA HBA drivers, and add support for PMP
to ahcisata(4) and siisata(4).

Thanks to:
- Protocase (http://www.protocase.com/) which provided a system
with lots of controllers, SATA PMP and drive slots
- Conservation Genomics Laboratory, Department of Biology, New Mexico State
University for hosting the above system
- Brook Milligan, who set up remote access and has been very responsive
when SATA cable move was needed
 1.15 15-May-2012  bouyer If the PHY reports link up but no communication, wait 5 more
seconds for communications to be established. This seems to be needed for
some configurations.
 1.14 12-Dec-2010  jakllsch branches: 1.14.8; 1.14.12;
Actually, 31244 does not refer to the Silicon Image SiI3124,
but the now-obscure Intel GD31244.
 1.13 11-Dec-2010  matt Fix typo (31224 -> 3124) and constify an array.
 1.12 18-Nov-2008  cegger branches: 1.12.8;
SATA GEN3 speed is 6.0Gb/s
 1.11 28-Apr-2008  martin branches: 1.11.6; 1.11.8; 1.11.10;
Remove clause 3 and 4 from TNF licenses
 1.10 18-Mar-2008  cube branches: 1.10.2; 1.10.4;
Split device_t and softc for ATA devices, as well as wd(4). Other
cosmetic changes where appropriate.
 1.9 11-Dec-2007  lukem branches: 1.9.4; 1.9.8;
use __KERNEL_RCSID()
 1.8 30-Nov-2006  bouyer branches: 1.8.24; 1.8.34; 1.8.36; 1.8.38;
It's 3.0Gb/s, not GB/s. Pointed out by Thomas E. Spanjaard.
 1.7 27-Nov-2006  dogcow revert last change, as havard's fix is more general.
 1.6 22-Nov-2006  dogcow #include <sys/device.h> to fix build failure on sparc64.
(why only sparc64? no idea.)
 1.5 22-Nov-2006  bouyer Cosmetic: remove one ":" from the port status string, it looks better this
way.
 1.4 22-Nov-2006  bouyer In sata_reset_interface() return only DET bits of SStatus, as the
return value is compared against SStatus_DET_*
 1.3 20-Nov-2006  bouyer Move part of wdc_sataprobe() to sys/dev/ata/sata_subr.c so that it can be
shared with non-wdc SATA controllers.
 1.2 11-Dec-2005  christos branches: 1.2.20; 1.2.22;
merge ktrace-lwp.
 1.1 28-May-2004  thorpej branches: 1.1.2; 1.1.14;
Add common routines for SATA controllers. Right now, we have sata_speed(),
which reports the speed encoded in the SStatus register.
 1.1.14.3 24-Mar-2008  yamt sync with head.
 1.1.14.2 21-Jan-2008  yamt sync with head
 1.1.14.1 30-Dec-2006  yamt sync with head.
 1.1.2.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.1.2.3 18-Sep-2004  skrll Sync with HEAD.
 1.1.2.2 03-Aug-2004  skrll Sync with HEAD
 1.1.2.1 28-May-2004  skrll file sata_subr.c was added on branch ktrace-lwp on 2004-08-03 10:45:46 +0000
 1.2.22.1 10-Dec-2006  yamt sync with head.
 1.2.20.1 12-Jan-2007  ad Sync with head.
 1.8.38.1 13-Dec-2007  bouyer Sync with HEAD
 1.8.36.1 11-Dec-2007  yamt sync with head.
 1.8.34.1 26-Dec-2007  ad Sync with head.
 1.8.24.2 23-Mar-2008  matt sync with HEAD
 1.8.24.1 09-Jan-2008  matt sync with HEAD
 1.9.8.3 17-Jan-2009  mjf Sync with HEAD.
 1.9.8.2 02-Jun-2008  mjf Sync with HEAD.
 1.9.8.1 03-Apr-2008  mjf Sync with HEAD.
 1.9.4.1 24-Mar-2008  keiichi sync with head.
 1.10.4.2 04-May-2009  yamt sync with head.
 1.10.4.1 16-May-2008  yamt sync with head.
 1.10.2.1 18-May-2008  yamt sync with head.
 1.11.10.1 22-Nov-2008  snj branches: 1.11.10.1.8;
Pull up following revision(s) (requested by cegger in ticket #99):
sys/dev/ata/sata_subr.c: revision 1.12
SATA GEN3 speed is 6.0Gb/s
 1.11.10.1.8.1 07-Jan-2011  matt small fixes (constification + comment type).
 1.11.8.1 19-Jan-2009  skrll Sync with HEAD.
 1.11.6.1 13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.12.8.1 05-Mar-2011  rmind sync with head
 1.14.12.1 02-Jun-2012  mrg sync to latest -current.
 1.14.8.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.14.8.2 30-Oct-2012  yamt sync with head
 1.14.8.1 23-May-2012  yamt sync with head.
 1.20.2.2 03-Dec-2017  jdolecek update from HEAD
 1.20.2.1 23-Jun-2013  tls resync from head
 1.21.26.1 11-May-2017  pgoyette Sync with HEAD
 1.21.24.1 19-Sep-2017  jdolecek replace all remaining tsleep()/wakeup() calls with condition variables, or
calls to ata_delay(), as appropriate; change ata_delay() to require the
channel lock on entry, and pass the lock to kpause() for unlocking while
sleeping
 1.21.12.1 28-Aug-2017  skrll Sync with HEAD
 1.22.2.1 24-Feb-2023  martin Pull up following revision(s) (requested by msaitoh in ticket #1805):

sys/dev/ata/satavar.h: revision 1.10
sys/dev/ata/sata_subr.c: revision 1.24

split the port status reporting to new function sata_interpret_det()
so it can be called separately from sata_reset_interface()

do not treat PHY offline as an error, it's pretty normal when there
is no device actually connected

debugging aid for PR kern/52372
 1.23.2.1 25-Jun-2018  pgoyette Sync with HEAD
 1.9 15-Aug-2019  gson Set the ATAPI "BYTE COUNT LIMIT" field in the SATA case like we
already do in the ATA case, to make NetBSD work with the virtual
SATA CD-ROM of "qemu-system-i386 -machine q35". Fixes PR kern/54389.
OK mlelstv.
 1.8 07-Oct-2017  jdolecek branches: 1.8.4; 1.8.10;
Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.7 22-Jul-2012  jakllsch branches: 1.7.2; 1.7.28;
SATA uses the head portion of the device register in the command protocol for
port multiplier register access. Thus, in the LBA48 case, pass the complete
r(hd|dh)_dh byte back and forth to the upper layers. (This is irrelevant in
the LBA28/CHS code path for what should be obvious reasons.)
 1.6 24-Jan-2012  jakllsch Rework struct ata_command to support LBA28 and LBA48-protocol commands.
Implement 28- and 48-bit command issuance and parameter read-back in the
various ATA host drivers. Add LBA28-protocol support to ATAIOCCOMMAND ioctl.
 1.5 07-Apr-2010  jakllsch branches: 1.5.2; 1.5.10; 1.5.14;
satafis:
- Add function to parse RDH FIS for use in implementing AT_READREG.
- Correct and clean up some structure definitions.
- Sprinkle a bit of const.
- Remove dependency on <dev/ic/wdcreg.h>, WDCTL_4BIT doesn't seem
to be specified by any recent ATA standard, and it seems to make
no difference in practice.
- Stop using WDSD_IBM, these bits have been obsolete since before SATA.

siisata:
- Implement AT_READREG.
- Reap now-unneeded <dev/ic/wdcreg.h>.
- Whitespace and slight debug code cleanup.
- Some possibly-uncessary code reordering.

ahcisata:
- Implement AT_READREG.
- Reap now-unneeded <dev/ic/wdcreg.h>.
 1.4 19-Oct-2009  bouyer branches: 1.4.2; 1.4.4;
Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !
 1.3 04-Jul-2009  jakllsch branches: 1.3.2;
Further corrections for siisata.

Reset ch_status and ch_error at the beginning of any operation.

The chip only writes the RDH FIS to the SRAM when the error bit is
set in a RDH FIS (per the datasheet). Thus, satafis_sdb_parse() is
replaced with satafis_rhd_parse(). A valid FIS type field seems to
not be written to the SRAM, so don't bother checking it for the
correct magic.

Additionally, append 2009 to copyright year set, and remove
obsolete CVS Id lines from when this was in my repository.
 1.2 17-Jun-2009  jakllsch branches: 1.2.2;
Use abstracted FIS code for ahcisata(4). Have the FIS code pre-zero the FIS.
 1.1 17-Jun-2009  jakllsch ahcisata(4) and siisata(4) use similar SATA FIS functions, share them.
 1.2.2.5 11-Aug-2010  yamt sync with head.
 1.2.2.4 11-Mar-2010  yamt sync with head
 1.2.2.3 18-Jul-2009  yamt sync with head.
 1.2.2.2 20-Jun-2009  yamt sync with head
 1.2.2.1 17-Jun-2009  yamt file satafis_subr.c was added on branch yamt-nfs-mp on 2009-06-20 07:20:20 +0000
 1.3.2.4 28-Sep-2009  snj Pull up following revision(s) (requested by jakllsch in ticket #1023):
sys/dev/ata/satafis_subr.c: revision 1.3
sys/dev/ata/satafisvar.h: revision 1.2
sys/dev/ic/siisata.c: revision 1.7
sys/dev/ic/siisatareg.h: revision 1.4
Further corrections for siisata.
Reset ch_status and ch_error at the beginning of any operation.
The chip only writes the RDH FIS to the SRAM when the error bit is
set in a RDH FIS (per the datasheet). Thus, satafis_sdb_parse() is
replaced with satafis_rhd_parse(). A valid FIS type field seems to
not be written to the SRAM, so don't bother checking it for the
correct magic.
Additionally, append 2009 to copyright year set, and remove
obsolete CVS Id lines from when this was in my repository.
 1.3.2.3 28-Sep-2009  snj Pull up following revision(s) (requested by jakllsch in ticket #1023):
sys/dev/ata/satafis_subr.c: revision 1.2
Have the FIS code pre-zero the FIS.
 1.3.2.2 28-Sep-2009  snj Pull up following revision(s) (requested by jakllsch in ticket #1023):
sys/conf/files: revision 1.947
sys/dev/ata/files.ata: revision 1.21
sys/dev/ata/satafis_subr.c: revision 1.1
sys/dev/ata/satafisreg.h: revision 1.1
sys/dev/ata/satafisvar.h: revision 1.1
ahcisata(4) and siisata(4) use similar SATA FIS functions, share them.
 1.3.2.1 04-Jul-2009  snj file satafis_subr.c was added on branch netbsd-5 on 2009-09-28 00:13:02 +0000
 1.4.4.1 30-May-2010  rmind sync with head
 1.4.2.1 30-Apr-2010  uebayasi Sync with HEAD.
 1.5.14.1 18-Feb-2012  mrg merge to -current.
 1.5.10.2 30-Oct-2012  yamt sync with head
 1.5.10.1 17-Apr-2012  yamt sync with head
 1.5.2.3 24-Mar-2014  matt Merge satafis changes from HEAD for ahcisata
 1.5.2.2 21-Apr-2010  matt sync to netbsd-5
 1.5.2.1 07-Apr-2010  matt file satafis_subr.c was added on branch matt-nb5-mips64 on 2010-04-21 00:27:34 +0000
 1.7.28.3 19-Jul-2017  jdolecek tighen and expand error handling, mostly for NCQ use cases:
- make retry timeout callout per xfer, i.e. retry separately
- zero whole bio struct on retry to avoid more stale state
- add a REQUEUE option, which doesn't bump retry count
- add ata_read_log_ext_ncq() for NCQ recovery
- adjust logic for activating xfers - allow next command only when
it's for same drive, several concurrent are only supported when HBA
and driver support FIS-based switching
- add new ata_timeout() which handles race between callout_stop()
and the invokation, add appropriate handling on deactivate/free paths
- stop using ch_status/ch_error in non-wdc code; later it will be dropped
completely
 1.7.28.2 19-Apr-2017  jdolecek adjust ata code to support more than one active command, including the
timeout handling, add support for NCQ commands

move probe for NCQ and number of tags to middle layer, negotiate mutual
support between drive and controller

implement NCQ support in ahci(4)
 1.7.28.1 15-Apr-2017  jdolecek make ata_xfer's allocated as part of ata_queue and make it include ata_bio;
they are pre-allocated on attach and ata_get_xfer() now never sleep, drop the
pool

modify wd(4) to file the bio requests using the xfers and hence
make it possible to have more than one active I/O request in flight;
ata_bio callback doesn't need to allocate any memory any more,
require it to never return ATACMD_TRY_AGAIN

move lp, badsect, multi from ata_bio to ata_drive_datas, as they are per-drive,
not per transfer

drop unused drv atac_claim_hw/atac_free_hw hooks, and also drop again ata_bio
c_hwslot
 1.7.2.1 03-Dec-2017  jdolecek update from HEAD
 1.8.10.1 05-Sep-2019  martin Pull up following revision(s) (requested by gson in ticket #179):

sys/dev/ata/satafis_subr.c: revision 1.9

Set the ATAPI "BYTE COUNT LIMIT" field in the SATA case like we
already do in the ATA case, to make NetBSD work with the virtual
SATA CD-ROM of "qemu-system-i386 -machine q35". Fixes PR kern/54389.

OK mlelstv.
 1.8.4.1 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.4 24-Jan-2012  jakllsch Rework struct ata_command to support LBA28 and LBA48-protocol commands.
Implement 28- and 48-bit command issuance and parameter read-back in the
various ATA host drivers. Add LBA28-protocol support to ATAIOCCOMMAND ioctl.
 1.3 02-Nov-2011  jakllsch branches: 1.3.4;
Drop trailing whitespace.
 1.2 07-Apr-2010  jakllsch branches: 1.2.2; 1.2.10;
satafis:
- Add function to parse RDH FIS for use in implementing AT_READREG.
- Correct and clean up some structure definitions.
- Sprinkle a bit of const.
- Remove dependency on <dev/ic/wdcreg.h>, WDCTL_4BIT doesn't seem
to be specified by any recent ATA standard, and it seems to make
no difference in practice.
- Stop using WDSD_IBM, these bits have been obsolete since before SATA.

siisata:
- Implement AT_READREG.
- Reap now-unneeded <dev/ic/wdcreg.h>.
- Whitespace and slight debug code cleanup.
- Some possibly-uncessary code reordering.

ahcisata:
- Implement AT_READREG.
- Reap now-unneeded <dev/ic/wdcreg.h>.
 1.1 17-Jun-2009  jakllsch branches: 1.1.2; 1.1.4; 1.1.6; 1.1.8;
ahcisata(4) and siisata(4) use similar SATA FIS functions, share them.
 1.1.8.1 30-May-2010  rmind sync with head
 1.1.6.1 30-Apr-2010  uebayasi Sync with HEAD.
 1.1.4.2 28-Sep-2009  snj Pull up following revision(s) (requested by jakllsch in ticket #1023):
sys/conf/files: revision 1.947
sys/dev/ata/files.ata: revision 1.21
sys/dev/ata/satafis_subr.c: revision 1.1
sys/dev/ata/satafisreg.h: revision 1.1
sys/dev/ata/satafisvar.h: revision 1.1
ahcisata(4) and siisata(4) use similar SATA FIS functions, share them.
 1.1.4.1 17-Jun-2009  snj file satafisreg.h was added on branch netbsd-5 on 2009-09-28 00:13:02 +0000
 1.1.2.3 11-Aug-2010  yamt sync with head.
 1.1.2.2 20-Jun-2009  yamt sync with head
 1.1.2.1 17-Jun-2009  yamt file satafisreg.h was added on branch yamt-nfs-mp on 2009-06-20 07:20:20 +0000
 1.2.10.2 17-Apr-2012  yamt sync with head
 1.2.10.1 10-Nov-2011  yamt sync with head
 1.2.2.3 24-Mar-2014  matt Merge satafis changes from HEAD for ahcisata
 1.2.2.2 21-Apr-2010  matt sync to netbsd-5
 1.2.2.1 07-Apr-2010  matt file satafisreg.h was added on branch matt-nb5-mips64 on 2010-04-21 00:27:34 +0000
 1.3.4.1 18-Feb-2012  mrg merge to -current.
 1.4 07-Oct-2017  jdolecek Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.3 07-Apr-2010  jakllsch branches: 1.3.2; 1.3.20; 1.3.50;
satafis:
- Add function to parse RDH FIS for use in implementing AT_READREG.
- Correct and clean up some structure definitions.
- Sprinkle a bit of const.
- Remove dependency on <dev/ic/wdcreg.h>, WDCTL_4BIT doesn't seem
to be specified by any recent ATA standard, and it seems to make
no difference in practice.
- Stop using WDSD_IBM, these bits have been obsolete since before SATA.

siisata:
- Implement AT_READREG.
- Reap now-unneeded <dev/ic/wdcreg.h>.
- Whitespace and slight debug code cleanup.
- Some possibly-uncessary code reordering.

ahcisata:
- Implement AT_READREG.
- Reap now-unneeded <dev/ic/wdcreg.h>.
 1.2 04-Jul-2009  jakllsch branches: 1.2.2; 1.2.4; 1.2.6;
Further corrections for siisata.

Reset ch_status and ch_error at the beginning of any operation.

The chip only writes the RDH FIS to the SRAM when the error bit is
set in a RDH FIS (per the datasheet). Thus, satafis_sdb_parse() is
replaced with satafis_rhd_parse(). A valid FIS type field seems to
not be written to the SRAM, so don't bother checking it for the
correct magic.

Additionally, append 2009 to copyright year set, and remove
obsolete CVS Id lines from when this was in my repository.
 1.1 17-Jun-2009  jakllsch branches: 1.1.2;
ahcisata(4) and siisata(4) use similar SATA FIS functions, share them.
 1.1.2.4 11-Aug-2010  yamt sync with head.
 1.1.2.3 18-Jul-2009  yamt sync with head.
 1.1.2.2 20-Jun-2009  yamt sync with head
 1.1.2.1 17-Jun-2009  yamt file satafisvar.h was added on branch yamt-nfs-mp on 2009-06-20 07:20:20 +0000
 1.2.6.1 30-May-2010  rmind sync with head
 1.2.4.1 30-Apr-2010  uebayasi Sync with HEAD.
 1.2.2.3 28-Sep-2009  snj Pull up following revision(s) (requested by jakllsch in ticket #1023):
sys/dev/ata/satafis_subr.c: revision 1.3
sys/dev/ata/satafisvar.h: revision 1.2
sys/dev/ic/siisata.c: revision 1.7
sys/dev/ic/siisatareg.h: revision 1.4
Further corrections for siisata.
Reset ch_status and ch_error at the beginning of any operation.
The chip only writes the RDH FIS to the SRAM when the error bit is
set in a RDH FIS (per the datasheet). Thus, satafis_sdb_parse() is
replaced with satafis_rhd_parse(). A valid FIS type field seems to
not be written to the SRAM, so don't bother checking it for the
correct magic.
Additionally, append 2009 to copyright year set, and remove
obsolete CVS Id lines from when this was in my repository.
 1.2.2.2 28-Sep-2009  snj Pull up following revision(s) (requested by jakllsch in ticket #1023):
sys/conf/files: revision 1.947
sys/dev/ata/files.ata: revision 1.21
sys/dev/ata/satafis_subr.c: revision 1.1
sys/dev/ata/satafisreg.h: revision 1.1
sys/dev/ata/satafisvar.h: revision 1.1
ahcisata(4) and siisata(4) use similar SATA FIS functions, share them.
 1.2.2.1 04-Jul-2009  snj file satafisvar.h was added on branch netbsd-5 on 2009-09-28 00:13:02 +0000
 1.3.50.1 19-Jul-2017  jdolecek tighen and expand error handling, mostly for NCQ use cases:
- make retry timeout callout per xfer, i.e. retry separately
- zero whole bio struct on retry to avoid more stale state
- add a REQUEUE option, which doesn't bump retry count
- add ata_read_log_ext_ncq() for NCQ recovery
- adjust logic for activating xfers - allow next command only when
it's for same drive, several concurrent are only supported when HBA
and driver support FIS-based switching
- add new ata_timeout() which handles race between callout_stop()
and the invokation, add appropriate handling on deactivate/free paths
- stop using ch_status/ch_error in non-wdc code; later it will be dropped
completely
 1.3.20.1 03-Dec-2017  jdolecek update from HEAD
 1.3.2.3 24-Mar-2014  matt Merge satafis changes from HEAD for ahcisata
 1.3.2.2 21-Apr-2010  matt sync to netbsd-5
 1.3.2.1 07-Apr-2010  matt file satafisvar.h was added on branch matt-nb5-mips64 on 2010-04-21 00:27:34 +0000
 1.16 13-Apr-2020  jdolecek fix use-after-free for ata xfer on bio submission found by KASAN

driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself

PR kern/55169 by Nick Hudson
 1.15 22-Oct-2018  jdolecek branches: 1.15.4; 1.15.10;
Merge jdolecek-ncqfixes branch

- ata_xfer's are dynamicall allocated as needed using a pool, no longer
limited to number of possible openings supported by controller; dump
and recovery paths use dedicated pre-allocated storage
- moved callouts and condvars from ata_xfer to queue or channel, so that
ata_xfer does not need special initialization
- slot allocation now done when xfer is being activated, uncoupled
from memory allocation; active slots are no longer tracked by controller
code
- channel and drive reset is done always via the atabus thread, and
now executes with channel locked the whole time
- NCQ recovery moved to shared function, and run via the thread also
- added some workarounds for buggy error recovery AHCI emulation in QEMU
and Parallels

designed to primarily fix kern/52614, but might also help with kern/47041
and kern/53183
 1.14 29-Jul-2018  jdolecek branches: 1.14.2;
mark satapmp_rescan() static, it's not to be used outside satapmp_subr.c
 1.13 07-Oct-2017  jdolecek branches: 1.13.2; 1.13.4;
Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.12 03-May-2013  jakllsch branches: 1.12.24;
Wait for DRDY after the PMP commands and don't wait before.
Should fix the worst part of PR kern/47793.
 1.11 17-Apr-2013  jakllsch Also print port multiplier port number when register accesses fail.
 1.10 01-Aug-2012  bouyer branches: 1.10.2; 1.10.4;
Make this compile again after DRIVET rename
 1.9 01-Aug-2012  bouyer Apply back changes that were reverted on Jul 24 and Jul 26 (general ata/wdc
cleanup and SATA PMP support), now that I'm back to fix the fallouts.
 1.8 26-Jul-2012  jakllsch Revert, with intention of restoring in a less invasive way, the SATA Port
Multiplier code.

ok christos@
 1.7 24-Jul-2012  jakllsch Revert dsl@'s changes of Sun, 15 Jul 2012 10:55:35 +0000 and
Sun, 15 Jul 2012 10:56:50 +0000, excepting the kernel version bump.
First step in reverting regressions to ata(4) subsystem during the addition of
port multiplier support.
 1.6 22-Jul-2012  jakllsch Port multiplier registers are 64-bits wide (although the ones we need only
implement 32-bits). Implement the access routines using 64-bit values and add
32-bit wrappers thereto.
 1.5 22-Jul-2012  jakllsch The vendor and product IDs are 16-bit, print out as such.
 1.4 22-Jul-2012  jakllsch Correctly issue port multiplier register access commands as LBA48 now that the
FIS code doesn't get in the way of doing that.
 1.3 22-Jul-2012  jakllsch KASSERT if we attempt to access an invalid port.
Also, small whitespace change for internal consistency.
 1.2 15-Jul-2012  dsl Some namespace protection (and add greppablity).
Prefix the DRIVE_ and DRIVET_ constants from atavar.h with ATA_.
Don't use an enum for drive_type - you don't know how big it will be.
Move driver_type to avoid implicit structure padding (esp on arm).
This change is purely lexical and mechanical.

Update to 6.99.9 - this wasn't done when the SATA PMP changes
were made - I'm sure they warranted a bump.
 1.1 02-Jul-2012  bouyer Add sata Port MultiPlier (PMP) support to the ata bus layer,
as described in
http://mail-index.netbsd.org/tech-kern/2012/06/23/msg013442.html
PMP support in integrated to the atabus layer.
struct ata_channel's ch_drive[] is not dynamically allocated, and ch_ndrive
(renamed to ch_ndrives) closely reflects the size of the ch_drive[] array.
Add helper functions atabus_alloc_drives() and atabus_free_drives()
to manage ch_drive[]/ch_ndrives.
Add wdc_maxdrives to struct wdc_softc so that bus front-end can specify
how much drive they really support (master/slave or single).
ata_reset_drive() callback gains a uint32_t *sigp argument which,
when not NULL, will contain the signature of the device being reset.
While there, some cosmetic changes:
- added a drive_type enum to ata_drive_datas, and stop encoding the
probed drive type in drive_flags (we were out of drive flags anyway).
- rename DRIVE_ATAPIST to DRIVE_ATAPIDSCW to better reflect what this
really is
- remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas
- factor out the interpretation of SATA signatures in sata_interpet_sig()

propagate these changes to the ATA HBA drivers, and add support for PMP
to ahcisata(4) and siisata(4).

Thanks to:
- Protocase (http://www.protocase.com/) which provided a system
with lots of controllers, SATA PMP and drive slots
- Conservation Genomics Laboratory, Department of Biology, New Mexico State
University for hosting the above system
- Brook Milligan, who set up remote access and has been very responsive
when SATA cable move was needed
 1.10.4.3 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.10.4.2 30-Oct-2012  yamt sync with head
 1.10.4.1 01-Aug-2012  yamt file satapmp_subr.c was added on branch yamt-pagecache on 2012-10-30 17:20:53 +0000
 1.10.2.2 03-Dec-2017  jdolecek update from HEAD
 1.10.2.1 23-Jun-2013  tls resync from head
 1.12.24.6 19-Sep-2017  jdolecek replace all remaining tsleep()/wakeup() calls with condition variables, or
calls to ata_delay(), as appropriate; change ata_delay() to require the
channel lock on entry, and pass the lock to kpause() for unlocking while
sleeping
 1.12.24.5 26-Jul-2017  jdolecek write the device and channel for port multiplier attach (not just the atabus),
so it's easier to check
 1.12.24.4 20-Jun-2017  jdolecek remove ata queue downsizing - every device, attached to the same channel,
uses slots according to it's own limits

wdc code changed to expect maximum one active xfer, and not check number
of openings in the channel; this is to facilitate using wdc functions
for e.g. handling of atapi commands for drivers which support both ATAPI
and NCQ
 1.12.24.3 19-Jun-2017  jdolecek add ata_channel lock, use it to protect queue manipulation (only that for now);
add ata_channel_detach() to destroy the locks

change ata_get_xfer() so that it can wait for xfer, convert all on-stack
xfer code to use the blocking variant

fix siisata_reset_drive() to use polled reset and not try ata_activate_xfer(),
convert drive probe code also over from slot0 XXX to ata_get_xfer()

drive reset and PMP now works on siisata(4) too; changes tested also
on piixide(4), ahci(4), mvsata(4)
 1.12.24.2 17-Jun-2017  jdolecek make PMP working great again

tested with mvsata(4), my ahcisata(4) controller unfortunately doesn't
support PMP
 1.12.24.1 15-Apr-2017  jdolecek pass also ata_command via ata_xfer, callers of ata_exec_command() is now
responsible for allocation/disposal of the structure

change code to allocate ata_xfer for commands on stack same way as previously
the ata_command were, using c_slot 0; adjust asserts so that it would allow
several xfers with same c_slot, as long as only one such transfer is active
at a time
 1.13.4.2 21-Apr-2020  martin Sync with HEAD
 1.13.4.1 10-Jun-2019  christos Sync with HEAD
 1.13.2.2 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.13.2.1 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.14.2.2 20-Oct-2018  jdolecek hold lock during drive reset (new locking protocol)

erase xfer before each reuse for read/write port commands
 1.14.2.1 22-Sep-2018  jdolecek separate ata_xfer slot allocation and the memory allocation, so that
there can be more queued xfers than number of supported slots by controller,
and use a pool instead of custom pre-allocation

primarily to help PR kern/52614

remove no longer needed custom wd(4) logic for flush cache

switch also wd(4) trim/suspend/setcache/wdioctlstrategy to sleep waiting
for the memory, they are all called from process context and this
avoids spurious failures
 1.15.10.1 20-Apr-2020  bouyer Sync with HEAD
 1.15.4.1 30-Dec-2022  martin Pull up following revision(s) (requested by tsutsui in ticket #1557):

sys/dev/ic/ahcisata_core.c: revision 1.83
sys/dev/ic/ahcisata_core.c: revision 1.102
sys/dev/ata/ata.c: revision 1.164
sys/dev/ata/ata_wdc.c: revision 1.115
sys/dev/ata/ata_recovery.c: revision 1.4
sys/dev/ic/siisata.c: revision 1.42
sys/dev/ic/wdc.c: revision 1.308
sys/dev/ic/mvsata.c: revision 1.56
sys/dev/scsipi/atapi_wdc.c: revision 1.138
sys/dev/ic/siisata.c: revision 1.49
sys/dev/ata/atavar.h: revision 1.105
sys/dev/ata/wd.c: revision 1.460
sys/dev/ata/ata.c: revision 1.155
sys/dev/ata/wd.c: revision 1.462
sys/dev/ata/atavar.h: revision 1.109
sys/dev/ata/satapmp_subr.c: revision 1.16
sys/dev/ic/wdc.c: revision 1.299
sys/dev/ic/ahcisata_core.c: revision 1.93
sys/dev/ata/ata_wdc.c: revision 1.120
sys/dev/ic/wdcvar.h: revision 1.100
sys/dev/scsipi/atapi_wdc.c: revision 1.141
sys/dev/ic/mvsata.c: revision 1.61
sys/dev/usb/umass_isdata.c (apply patch)

drop wd lock in wdstart1() before calling the ata_bio hook; when called
from ata thread context, that can still need to sleep for wdc attachments
in wdcwait()

fix use-after-free for ata xfer on bio submission found by KASAN
driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself
PR kern/55169 by Nick Hudson

Function declaration formating whitespace consistency. NFCI.

PR kern/56403
Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.
(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.
Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().
(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).
"Go ahead" by jdolecek@.
 1.6 31-Jul-2012  bouyer Apply back changes that were reverted on Jul 24 and Jul 26 (general ata/wdc
cleanup and SATA PMP support), now that I'm back to fix the fallouts.
 1.5 26-Jul-2012  jakllsch Revert, with intention of restoring in a less invasive way, the SATA Port
Multiplier code.

ok christos@
 1.4 02-Jul-2012  bouyer Add sata Port MultiPlier (PMP) support to the ata bus layer,
as described in
http://mail-index.netbsd.org/tech-kern/2012/06/23/msg013442.html
PMP support in integrated to the atabus layer.
struct ata_channel's ch_drive[] is not dynamically allocated, and ch_ndrive
(renamed to ch_ndrives) closely reflects the size of the ch_drive[] array.
Add helper functions atabus_alloc_drives() and atabus_free_drives()
to manage ch_drive[]/ch_ndrives.
Add wdc_maxdrives to struct wdc_softc so that bus front-end can specify
how much drive they really support (master/slave or single).
ata_reset_drive() callback gains a uint32_t *sigp argument which,
when not NULL, will contain the signature of the device being reset.
While there, some cosmetic changes:
- added a drive_type enum to ata_drive_datas, and stop encoding the
probed drive type in drive_flags (we were out of drive flags anyway).
- rename DRIVE_ATAPIST to DRIVE_ATAPIDSCW to better reflect what this
really is
- remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas
- factor out the interpretation of SATA signatures in sata_interpet_sig()

propagate these changes to the ATA HBA drivers, and add support for PMP
to ahcisata(4) and siisata(4).

Thanks to:
- Protocase (http://www.protocase.com/) which provided a system
with lots of controllers, SATA PMP and drive slots
- Conservation Genomics Laboratory, Department of Biology, New Mexico State
University for hosting the above system
- Brook Milligan, who set up remote access and has been very responsive
when SATA cable move was needed
 1.3 28-Apr-2008  martin branches: 1.3.34;
Remove clause 3 and 4 from TNF licenses
 1.2 11-Dec-2005  christos branches: 1.2.70; 1.2.72; 1.2.74;
merge ktrace-lwp.
 1.1 21-Aug-2004  thorpej branches: 1.1.2;
Register description for SATA port multipliers.
 1.1.2.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.1.2.3 18-Sep-2004  skrll Sync with HEAD.
 1.1.2.2 25-Aug-2004  skrll Sync with HEAD.
 1.1.2.1 21-Aug-2004  skrll file satapmpreg.h was added on branch ktrace-lwp on 2004-08-25 06:57:34 +0000
 1.2.74.1 16-May-2008  yamt sync with head.
 1.2.72.1 18-May-2008  yamt sync with head.
 1.2.70.1 02-Jun-2008  mjf Sync with HEAD.
 1.3.34.1 30-Oct-2012  yamt sync with head
 1.5 29-Jul-2018  jdolecek mark satapmp_rescan() static, it's not to be used outside satapmp_subr.c
 1.4 07-Oct-2017  jdolecek branches: 1.4.2; 1.4.4;
Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.3 01-Aug-2012  bouyer branches: 1.3.2; 1.3.4; 1.3.30;
Apply back changes that were reverted on Jul 24 and Jul 26 (general ata/wdc
cleanup and SATA PMP support), now that I'm back to fix the fallouts.
 1.2 26-Jul-2012  jakllsch Revert, with intention of restoring in a less invasive way, the SATA Port
Multiplier code.

ok christos@
 1.1 02-Jul-2012  bouyer Add sata Port MultiPlier (PMP) support to the ata bus layer,
as described in
http://mail-index.netbsd.org/tech-kern/2012/06/23/msg013442.html
PMP support in integrated to the atabus layer.
struct ata_channel's ch_drive[] is not dynamically allocated, and ch_ndrive
(renamed to ch_ndrives) closely reflects the size of the ch_drive[] array.
Add helper functions atabus_alloc_drives() and atabus_free_drives()
to manage ch_drive[]/ch_ndrives.
Add wdc_maxdrives to struct wdc_softc so that bus front-end can specify
how much drive they really support (master/slave or single).
ata_reset_drive() callback gains a uint32_t *sigp argument which,
when not NULL, will contain the signature of the device being reset.
While there, some cosmetic changes:
- added a drive_type enum to ata_drive_datas, and stop encoding the
probed drive type in drive_flags (we were out of drive flags anyway).
- rename DRIVE_ATAPIST to DRIVE_ATAPIDSCW to better reflect what this
really is
- remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas
- factor out the interpretation of SATA signatures in sata_interpet_sig()

propagate these changes to the ATA HBA drivers, and add support for PMP
to ahcisata(4) and siisata(4).

Thanks to:
- Protocase (http://www.protocase.com/) which provided a system
with lots of controllers, SATA PMP and drive slots
- Conservation Genomics Laboratory, Department of Biology, New Mexico State
University for hosting the above system
- Brook Milligan, who set up remote access and has been very responsive
when SATA cable move was needed
 1.3.30.1 19-Sep-2017  jdolecek replace all remaining tsleep()/wakeup() calls with condition variables, or
calls to ata_delay(), as appropriate; change ata_delay() to require the
channel lock on entry, and pass the lock to kpause() for unlocking while
sleeping
 1.3.4.2 30-Oct-2012  yamt sync with head
 1.3.4.1 01-Aug-2012  yamt file satapmpvar.h was added on branch yamt-pagecache on 2012-10-30 17:20:53 +0000
 1.3.2.1 03-Dec-2017  jdolecek update from HEAD
 1.4.4.1 10-Jun-2019  christos Sync with HEAD
 1.4.2.1 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.7 16-Feb-2022  andvar fix various typos, mainly in comments.
 1.6 27-Dec-2020  jmcneill Add G3 and DevSleep definitions. This changes the mask used by
SControl_IPM_NONE from 0x3 to 0x7.
 1.5 28-Apr-2008  martin branches: 1.5.94; 1.5.102;
Remove clause 3 and 4 from TNF licenses
 1.4 11-Dec-2005  christos branches: 1.4.70; 1.4.72; 1.4.74;
merge ktrace-lwp.
 1.3 23-May-2004  wiz branches: 1.3.2;
Fix typo reported by Alexander Yurchenko grange at rt mipt ru in private
mail.
 1.2 17-Dec-2003  thorpej Some additional SATA-II registers.
 1.1 14-Dec-2003  thorpej Serial ATA register definitions.
 1.3.2.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.3.2.3 18-Sep-2004  skrll Sync with HEAD.
 1.3.2.2 03-Aug-2004  skrll Sync with HEAD
 1.3.2.1 23-May-2004  skrll file satareg.h was added on branch ktrace-lwp on 2004-08-03 10:45:46 +0000
 1.4.74.1 16-May-2008  yamt sync with head.
 1.4.72.1 18-May-2008  yamt sync with head.
 1.4.70.1 02-Jun-2008  mjf Sync with HEAD.
 1.5.102.1 03-Jan-2021  thorpej Sync w/ HEAD.
 1.5.94.1 30-Dec-2020  martin Pull up following revision(s) (requested by jmcneill in ticket #1167):

sys/dev/ic/ahcisata_core.c: revision 1.84
sys/dev/ic/ahcisata_core.c: revision 1.85
sys/dev/ic/ahcisata_core.c: revision 1.88
sys/dev/ic/ahcisata_core.c: revision 1.89
sys/arch/arm/nvidia/tegra_ahcisata.c: revision 1.13
sys/dev/ic/ahcisatavar.h: revision 1.26
sys/dev/ic/ahcisata_core.c: revision 1.90
sys/dev/ic/ahcisata_core.c: revision 1.91
sys/dev/ic/ahcisata_core.c: revision 1.92
sys/dev/ata/satareg.h: revision 1.6

ahci_exec_fis: wait for the correct amount of time when AT_WAIT is set

Retry clearing WDCTL_RST a few times before giving up. Makes SATA work in
Solidrun Honeycomb LX2K.

AHCI 1.3.1 specification says that it is good practice for system software
to 'zero-out' the memory allocated and referenced by PxCLB and PxFB.

ahci_intr: use ffs in the port bitmask instead of looping over all 32 bits

AHCI 1.3.1 section 5.5.3 "Processing Completed Commands" says that we
should clear PxIS before IS.IPS.

Add G3 and DevSleep definitions. This changes the mask used by
SControl_IPM_NONE from 0x3 to 0x7.

Make sure to ack IS after PxIS when polling and when using multiple MSI-X
messages.

Remove the AHCI_QUIRK_SKIP_RESET quirk now that the underlying issue is
fixed.
 1.10 21-Jun-2018  jdolecek split the port status reporting to new function sata_interpret_det()
so it can be called separately from sata_reset_interface()

do not treat PHY offline as an error, it's pretty normal when there
is no device actually connected

debugging aid for PR kern/52372
 1.9 03-Apr-2013  bouyer branches: 1.9.28; 1.9.34;
Fix kernel dump on ahci controller, by making sure we won't sleep
while dumping:
- introduce ata_delay() which either use delay() or kpause()
depending on flags. use it in sata_reset_interface() and
some ahci functions
- kill ATA_NOSLEEP, it was tested but never set. use ATA_POLL instead.
- reduce delay while polling in ahci, to speed up the dump

Should fix PR kern/41095
 1.8 31-Jul-2012  bouyer branches: 1.8.2;
Apply back changes that were reverted on Jul 24 and Jul 26 (general ata/wdc
cleanup and SATA PMP support), now that I'm back to fix the fallouts.
 1.7 26-Jul-2012  jakllsch Revert, with intention of restoring in a less invasive way, the SATA Port
Multiplier code.

ok christos@
 1.6 02-Jul-2012  bouyer Add sata Port MultiPlier (PMP) support to the ata bus layer,
as described in
http://mail-index.netbsd.org/tech-kern/2012/06/23/msg013442.html
PMP support in integrated to the atabus layer.
struct ata_channel's ch_drive[] is not dynamically allocated, and ch_ndrive
(renamed to ch_ndrives) closely reflects the size of the ch_drive[] array.
Add helper functions atabus_alloc_drives() and atabus_free_drives()
to manage ch_drive[]/ch_ndrives.
Add wdc_maxdrives to struct wdc_softc so that bus front-end can specify
how much drive they really support (master/slave or single).
ata_reset_drive() callback gains a uint32_t *sigp argument which,
when not NULL, will contain the signature of the device being reset.
While there, some cosmetic changes:
- added a drive_type enum to ata_drive_datas, and stop encoding the
probed drive type in drive_flags (we were out of drive flags anyway).
- rename DRIVE_ATAPIST to DRIVE_ATAPIDSCW to better reflect what this
really is
- remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas
- factor out the interpretation of SATA signatures in sata_interpet_sig()

propagate these changes to the ATA HBA drivers, and add support for PMP
to ahcisata(4) and siisata(4).

Thanks to:
- Protocase (http://www.protocase.com/) which provided a system
with lots of controllers, SATA PMP and drive slots
- Conservation Genomics Laboratory, Department of Biology, New Mexico State
University for hosting the above system
- Brook Milligan, who set up remote access and has been very responsive
when SATA cable move was needed
 1.5 28-Apr-2008  martin branches: 1.5.34;
Remove clause 3 and 4 from TNF licenses
 1.4 19-Oct-2007  ad branches: 1.4.16; 1.4.18; 1.4.20;
machine/{bus,cpu,intr}.h -> sys/{bus,cpu,intr}.h
 1.3 20-Nov-2006  bouyer branches: 1.3.8; 1.3.22; 1.3.24; 1.3.28;
Move part of wdc_sataprobe() to sys/dev/ata/sata_subr.c so that it can be
shared with non-wdc SATA controllers.
 1.2 11-Dec-2005  christos branches: 1.2.20; 1.2.22;
merge ktrace-lwp.
 1.1 28-May-2004  thorpej branches: 1.1.2; 1.1.14;
Add common routines for SATA controllers. Right now, we have sata_speed(),
which reports the speed encoded in the SStatus register.
 1.1.14.2 27-Oct-2007  yamt sync with head.
 1.1.14.1 30-Dec-2006  yamt sync with head.
 1.1.2.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.1.2.3 18-Sep-2004  skrll Sync with HEAD.
 1.1.2.2 03-Aug-2004  skrll Sync with HEAD
 1.1.2.1 28-May-2004  skrll file satavar.h was added on branch ktrace-lwp on 2004-08-03 10:45:46 +0000
 1.2.22.1 10-Dec-2006  yamt sync with head.
 1.2.20.1 12-Jan-2007  ad Sync with head.
 1.3.28.1 25-Oct-2007  bouyer Sync with HEAD.
 1.3.24.1 06-Nov-2007  matt sync with HEAD
 1.3.22.1 26-Oct-2007  joerg Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.
 1.3.8.1 23-Oct-2007  ad Sync with head.
 1.4.20.1 16-May-2008  yamt sync with head.
 1.4.18.1 18-May-2008  yamt sync with head.
 1.4.16.1 02-Jun-2008  mjf Sync with HEAD.
 1.5.34.2 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.5.34.1 30-Oct-2012  yamt sync with head
 1.8.2.1 23-Jun-2013  tls resync from head
 1.9.34.1 25-Jun-2018  pgoyette Sync with HEAD
 1.9.28.1 24-Feb-2023  martin Pull up following revision(s) (requested by msaitoh in ticket #1805):

sys/dev/ata/satavar.h: revision 1.10
sys/dev/ata/sata_subr.c: revision 1.24

split the port status reporting to new function sata_interpret_det()
so it can be called separately from sata_reset_interface()

do not treat PHY offline as an error, it's pretty normal when there
is no device actually connected

debugging aid for PR kern/52372
 1.474 13-Apr-2025  jakllsch Add physical sector and alignment info to struct disk_geom and the
geometry plist, and handle in partutil.

Bump version for disk_geom addition.

Collect DIOCGSECTORALIGN handling into one place.
 1.473 27-Feb-2025  jakllsch Ensure dsa_firstaligned returned from DIOCGSECTORALIGN is actually the first
 1.472 23-Feb-2025  jakllsch atap_logical_align is already what we want, we don't have to subtract it
from the logical blocks per physical block number.
 1.471 17-Feb-2025  jakllsch Fix aprint_error_dev() abuse
 1.470 22-Sep-2024  uwe wd.c: consistent whitespace within one #define group
 1.469 05-Feb-2024  andvar branches: 1.469.2;
fix various typos in comments.
 1.468 24-Jan-2023  mlelstv Sanitize capacity values.
 1.467 28-Mar-2022  riastradh wd(4): Use d_cfdriver/devtounit to avoid open/detach races.
 1.466 28-Dec-2021  riastradh wd(4): Fix bugs in softbadsect handling.

- Don't copyout kernel virtual addresses (of SLIST entries) that
userland won't use anyway.
=> The structure still has space for this pointer; it's just always
null when userland gets it now.

- Don't copyout under a lock.

- Stop and return error if copyout fails (unless we've already copied
some out).

- Don't kmem_free under a lock.

XXX Unclear whether anyone actually uses WD_SOFTBADSECT or why --
it's always been disabled by default. Maybe we should just remove
it?
 1.465 28-Sep-2020  jakllsch fix typo that prevented bytes/physsect reporting from working
 1.464 27-Sep-2020  christos de-quadruplicate, remove unused argument
 1.463 24-May-2020  jdolecek account for already transferred data (partially done I/O) when
retrying an xfer, to avoid reading/writing data from/to wrong offset,
and eventually beyond the end of data buffer

fixes data corruption under QEMU observed by Paul Ripke for emulated
IDE drives
 1.462 13-Apr-2020  jdolecek fix use-after-free for ata xfer on bio submission found by KASAN

driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself

PR kern/55169 by Nick Hudson
 1.461 13-Apr-2020  maxv constify
 1.460 07-Apr-2020  jdolecek branches: 1.460.2;
drop wd lock in wdstart1() before calling the ata_bio hook; when called
from ata thread context, that can still need to sleep for wdc attachments
in wdcwait()
 1.459 02-Mar-2020  riastradh New ioctl DIOCGSECTORALIGN returns sector alignment parameters.

struct disk_sectoralign {
/* First aligned sector number. */
uint32_t dsa_firstaligned;

/* Number of sectors per aligned unit. */
uint32_t dsa_alignment;
};

- Teach wd(4) to get it from ATA.
- Teach cgd(4) to pass it through from the underlying disk.
- Teach dk(4) to pass it through with adjustments.
- Teach zpool (zfs) to take advantage of it.
=> XXX zpool doesn't seem to understand when the vdev's starting
sector is misaligned.

Missing:

- ccd(4) and raidframe(4) support -- these should support _using_
DIOCGSECTORALIGN to decide where to start putting ccd or raid
stripes on disk, and these should perhaps _implement_
DIOCGSECTORALIGN by reporting the stripe/interleave factor.

- sd(4) support -- I don't know any obvious way to get it from SCSI,
but if any SCSI wizards know better than I, please feel free to
teach sd(4) about it!

- any ld(4) attachments -- might be worth teaching the ld drivers for
nvme and various raid controllers to get the aligned sector size

There's some duplicate logic here for now. I'm doing it this way,
rather than gathering the logic into a new disklabel_sectoralign
function or something, so that this change is limited to adding a new
ioctl, without any new kernel symbols, in order to make it easy to
pull up to netbsd-9 without worrying about the module ABI.
 1.458 01-Mar-2020  riastradh Add a flag to dk_dump for virtual disk devices.

If a disk is backed by a physical medium other than itself, such as
cgd(4), then it passes DK_DUMP_RECURSIVE to disable the recursion
detection for dk_dump.

If, however, a device represents a physical medium on its own, such
as wd(4), then it passes 0 instead.

With this, I can now dump to dk on cgd on dk on wd.
 1.457 18-Jan-2020  simonb Tidy quirk table and remove outdated quick from the quirk format string.
 1.456 18-Jan-2020  simonb Revert kern/54790 and kern/54855 NCQ fix that penalised all Samsung
EVO 860 drives.

ok jdolecek@
 1.455 14-Jan-2020  jdolecek enable the BAD_NCQ quirk for all 860 EVO drives

XXX work-in-progress, it's not clear whether this is driver or controller
XXX problem
 1.454 13-Jan-2020  jdolecek disable NCQ by default for "Samsung SSD 860 EVO 1TB" and
"Samsung SSD 860 EVO 500GB" - these drives have known broken NCQ support
particularly when used with AMD SB710/750 chipsets, problem occur also
under Linux and Windows

https://eu.community.samsung.com/t5/Cameras-IT-Everything-Else/860-EVO-250GB-causing-freezes-on-AMD-system/td-p/575813
https://bugzilla.kernel.org/show_bug.cgi?id=201693

It seems there is no Samsung firmware update to fix this even.

Disable NCQ regardless of the controller, it's likely same problem
exists with other controllers too.

This should fix PR kern/54790 and PR kern/54855
 1.453 27-Dec-2019  msaitoh branches: 1.453.2;
s/transfered/transferred/
 1.452 06-Jun-2019  mlelstv branches: 1.452.2;
Take channel lock for calling reset_drive.
Should fix PR 54217.
 1.451 06-Jun-2019  mlelstv Count only the initial start of a transfer, not the retries.
Should fix kern/54166.

Thanks to macallan@ for spotting the issue.
 1.450 24-May-2019  mlelstv Also schedule timeouts when all openings are in use.
 1.449 07-Apr-2019  bouyer Really implement WDF_DIRTY. patch(1) did something silly here ...
 1.448 05-Apr-2019  bouyer drop AT_RST_NOCMD, it's a cut'n'paste side effect
 1.447 05-Apr-2019  bouyer Implement a DIRTY flag (copied from sd(4)) so avoid flushing the cache if
there has been no write. This avoids a (long) timeout on the flush cache
command triggered by atactl sleep, when the device is open only by the atactl
command itself.
If a drive has no partition open and goes to sleep, the WDF_LOADED
flag is clear, and the next open will issue wd_get_params() command.
But to wake up the drive a reset is required, and wd_get_params() doens't
issue a reset on timeout, so there's no way to wake up the disk.
Add a retry after reset to wd_get_params().

Tested by Hauke Fath; fixes PR kern/49457
 1.446 19-Mar-2019  mlelstv The NCQ support added a private request queue to the wd driver. This
makes the regular buffer queue ineffective, it also allowed to queue
an unlimited number of requests.

Fix this by limiting the number of requests queued to the driver to
the possible number of concurrent NCQ transactions.
 1.445 19-Mar-2019  mlelstv Set disk model name as disk type. The information can be queried through
drvctl(4).
 1.444 19-Mar-2019  mlelstv Move standby on detach after wedges deletion in case wedges trigger
I/O on the parent disk. Add debug messages.
 1.443 24-Oct-2018  jdolecek move the comment (and expand) about NCQ TRIM to wd_trim()
 1.442 22-Oct-2018  jdolecek Merge jdolecek-ncqfixes branch

- ata_xfer's are dynamicall allocated as needed using a pool, no longer
limited to number of possible openings supported by controller; dump
and recovery paths use dedicated pre-allocated storage
- moved callouts and condvars from ata_xfer to queue or channel, so that
ata_xfer does not need special initialization
- slot allocation now done when xfer is being activated, uncoupled
from memory allocation; active slots are no longer tracked by controller
code
- channel and drive reset is done always via the atabus thread, and
now executes with channel locked the whole time
- NCQ recovery moved to shared function, and run via the thread also
- added some workarounds for buggy error recovery AHCI emulation in QEMU
and Parallels

designed to primarily fix kern/52614, but might also help with kern/47041
and kern/53183
 1.441 10-Aug-2018  jdolecek branches: 1.441.2;
fix race in wd_lastclose() on systems with two ide disks on same
channel, which happened when one disk had pending I/O while the other
disk executed the final disk flush - need to restart bufq processing
once xfer is freed in this case

it could happen e.g. on boot when system executes fsck on different
partitions on the two drives in parallell and hence open and closes
the disk devices repeatedly

add KASSERT() for empty bufq on wd_lastclose(), and fix similar issue
also on suspend/standby path

this was introduced by the NCQ merge and not dksubr - before the merge
each drive had their own xfer, so they could not block each other

fixes PR kern/52783 by Onno van der Linden; many thanks for extensive
help with tracking this down
 1.440 06-Aug-2018  jdolecek add wddebug() which dumps some status for attached disks; indended for
debugging of PR kern/52783
 1.439 03-Jun-2018  jdolecek branches: 1.439.2;
take mutex around check for pending flush, as the code before dksubr
conversion had, to avoid possible race

on my system doesn't really change behaviour, besides the test runs
being slightly faster (3x parallell pkgsrc archive extraction, up
to 5% difference), thought that can just be noise

done as part of investigation for PR kern/53183 by Sevan Janiyan
 1.438 07-Jan-2018  mlelstv branches: 1.438.2;
Fix block address calculation for bad sectors.
 1.437 13-Dec-2017  pgoyette Fix build for WD_SOFTBADSECT option. PR kern/52814

XXX No clue if this option actually works. This fix just makes it
XXX compile without error.
 1.436 07-Nov-2017  mlelstv Make wddone poll all drives of a channel again.
 1.435 03-Nov-2017  mlelstv Add WDF_OPEN flag to really disallow opening of a disk that has been invalidated.

Restore wdbiorestart function to actually retry the failed I/O request instead
of just restarting the queue.

Fix compilation without ATADEBUG.
 1.434 01-Nov-2017  mlelstv refactor wd and ataraid drivers to use common disk subroutines.
 1.433 19-Oct-2017  jdolecek more detailed debug info; also sync DEBUG_* values in wd.c with ata.c
 1.432 14-Oct-2017  jdolecek do not use the NCQ priority by default; seems it negatively affects
performance at least with some drives, so this needs better understood first
 1.431 14-Oct-2017  jdolecek only call drive reset with AT_POLL when the command itself was
polled, so that the logic for AT_POLL matches how e.g. ata_dmaerr() is
called; this was the original intent of the change in 1.428.2.25,
to make the error handling safe wrt. polled xfers

this is stopgap fix for ATA channel wedge after DMA error, as reported
by Martin Husemann in PR kern/52606, and PR kern/52605

problem happened due to ata_reset_channel() being called once in ata_dmaerr()
with flags == 0, which freezed channel and set flag to reset via thread,
then ata_reset_channel() was called via wdc_drive_reset() with AT_POLL, which
just executed the reset and cleared the flag, without clearing the extra
freeze; that logic will be refactored in separate commit
 1.430 07-Oct-2017  jdolecek Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.429 24-Apr-2017  jdolecek branches: 1.429.2;
remove the workaround for Seagate 'mod15write' bug, now driver only prints
error on boog; unfortunately the code actually doesn't work, and there is
little point trying to fix
 1.428 05-Mar-2017  mlelstv branches: 1.428.2;
Enhance disk metrics by calculating a weighted sum that is incremented
by the number of concurrent I/O requests. Also introduce a new disk_wait()
function to measure requests waiting in a bufq.
iostat -y now reports data about waiting and active requests.

So far only drivers using dksubr and dk, ccd, wd and xbd collect data about
waiting requests.
 1.427 20-Nov-2016  pgoyette branches: 1.427.2;
Avoid calling bufq_free() from critical section.
 1.426 05-Aug-2016  christos CID 1364758: Integer handling issues, avoid sign extension to 64 bits.
 1.425 22-Jul-2016  jakllsch Space before tab and trailing whitespace fixes.
 1.424 22-Jul-2016  jakllsch Add ATA8-ACS Long Logical Sector Feature Set support to wd(4).
 1.423 21-Jul-2016  jakllsch Call wd_params_to_properties() after softc is sufficently
initialized.
 1.422 21-Jul-2016  jakllsch Remove unused 'params' argument of local function wd_params_to_properties()
 1.421 19-Jun-2016  bouyer branches: 1.421.2;
Add a new config_detach() flag, DETACH_POWEROFF, which is set when
detaching devices at shutdown time with RB_POWERDOWN.
When detaching wd(4), put the drive in standby before detach
for DETACH_POWEROFF.
Fix PR kern/51252
 1.420 26-Apr-2015  mlelstv Use C99-style initializers for struct dkdriver.
 1.419 13-Apr-2015  riastradh Convert sys/dev to use <sys/rndsource.h>.
 1.418 02-Jan-2015  christos We have three sets of DTYPE_ constants in the kernel:
altq Drop Type
disklabel Disk Type
file Descriptor Type
(not to mention constants that contain the string DTYPE).
Let's make them two, by changing the disklabel one to be DisK TYPE since the
other disklabel constants seem to do that. Not many userland programs use
these constants (and the ones that they do are mostly in ifdefs). They will
be fixed shortly.
 1.417 31-Dec-2014  christos make more drivers use disk_ioctl, and add a dev parameter to it so that
we can merge the "easy" disklabel ioctls to it. Ultimately all this will
go do dk_ioctl once all the drivers have been converted.
 1.416 31-Dec-2014  christos Centralize wedge ioctls in disk_ioctl.
 1.415 04-Nov-2014  mlelstv branches: 1.415.2;
support DIOCMWEDGES ioctl.
 1.414 11-Oct-2014  mlelstv The partition size is always computed in native blocks. The code also assumes
that native blocks are always DEVB_SIZE (a few lines earlier) which makes
the current calculation a no-op.
 1.413 10-Sep-2014  matt g/u_int[0-9]*_t/ s/u_int/uint/g
 1.412 10-Aug-2014  tls branches: 1.412.2;
Merge tls-earlyentropy branch into HEAD.
 1.411 25-Jul-2014  dholland Drop the old discard/trim ioctls from wd.c.
 1.410 25-Jul-2014  dholland Implement d_discard for wd.
 1.409 25-Jul-2014  dholland Add d_discard to all struct cdevsw instances I could find.

All have been set to "nodiscard"; some should get a real implementation.
 1.408 25-Jul-2014  dholland Add d_discard to all struct bdevsw instances I could find.

I've set them all to nodiscard. Some of them (wd, dk, vnd, ld,
raidframe, maybe cgd) should be implemented for real.
 1.407 16-Mar-2014  dholland branches: 1.407.2;
Change (mostly mechanically) every cdevsw/bdevsw I can find to use
designated initializers.

I have not built every extant kernel so I have probably broken at
least one build; however I've also found and fixed some wrong
cdevsw/bdevsw entries so even if so I think we come out ahead.
 1.406 30-Oct-2013  drochner play the addref/delref game on suspend, prevents crash if the disk/CF Card
is eg. in a PCMCIA adapter and not mounted
 1.405 19-Aug-2013  skrll Don't return EPASSTHROUGH when succeeding in ODIOCGDINFO or ODIOCGDEFLABEL
 1.404 06-Aug-2013  soren TRIM showed up sometime between ATA-7 and ATA-8.
 1.403 29-May-2013  christos branches: 1.403.2;
phase 1 of disk geometry cleanup:
- centralize the geometry -> plist code so that we don't have
n useless copies of it.
 1.402 09-Jan-2013  riastradh Initialize b_resid before biodone in wdioctlstrategy error branch.

Otherwise we may panic when physio's biodone callback kasserts
something about b_resid, since nothing will have initialized it.
 1.401 19-Oct-2012  drochner Implement experimental support to pass notifications that a file
was deleted from the filesystem to the disk driver, commonly
known as "discard" or "trim".
fs/driver support is in ffs and ata wd for now.
This is what was posted here:
http://mail-index.netbsd.org/tech-kern/2012/02/28/msg012813.html
with minor cleanup, and the global switch replaced by a mount option.
 1.400 31-Jul-2012  bouyer branches: 1.400.2;
Apply back changes that were reverted on Jul 24 and Jul 26 (general ata/wdc
cleanup and SATA PMP support), now that I'm back to fix the fallouts.
 1.399 26-Jul-2012  jakllsch Revert, with intention of restoring in a less invasive way, the SATA Port
Multiplier code.

ok christos@
 1.398 24-Jul-2012  jakllsch Revert dsl@'s changes of Sun, 15 Jul 2012 10:55:35 +0000 and
Sun, 15 Jul 2012 10:56:50 +0000, excepting the kernel version bump.
First step in reverting regressions to ata(4) subsystem during the addition of
port multiplier support.
 1.397 22-Jul-2012  jakllsch FLUSH CACHE EXT is a 48-bit command, issue as such. Both FLUSH CACHE commands
will return the LBA of first failure in the return task file, specifiy that we
want it read, even if we don't look at the value.
 1.396 22-Jul-2012  jakllsch ATA-6 and ACS-2 drafts both say the flush cache commands may take longer than
30 seconds to complete. For now assume 5 minutes will always be enough.
 1.395 15-Jul-2012  dsl Some namespace protection (and add greppablity).
Prefix the DRIVE_ and DRIVET_ constants from atavar.h with ATA_.
Don't use an enum for drive_type - you don't know how big it will be.
Move driver_type to avoid implicit structure padding (esp on arm).
This change is purely lexical and mechanical.

Update to 6.99.9 - this wasn't done when the SATA PMP changes
were made - I'm sure they warranted a bump.
 1.394 02-Jul-2012  bouyer Add sata Port MultiPlier (PMP) support to the ata bus layer,
as described in
http://mail-index.netbsd.org/tech-kern/2012/06/23/msg013442.html
PMP support in integrated to the atabus layer.
struct ata_channel's ch_drive[] is not dynamically allocated, and ch_ndrive
(renamed to ch_ndrives) closely reflects the size of the ch_drive[] array.
Add helper functions atabus_alloc_drives() and atabus_free_drives()
to manage ch_drive[]/ch_ndrives.
Add wdc_maxdrives to struct wdc_softc so that bus front-end can specify
how much drive they really support (master/slave or single).
ata_reset_drive() callback gains a uint32_t *sigp argument which,
when not NULL, will contain the signature of the device being reset.
While there, some cosmetic changes:
- added a drive_type enum to ata_drive_datas, and stop encoding the
probed drive type in drive_flags (we were out of drive flags anyway).
- rename DRIVE_ATAPIST to DRIVE_ATAPIDSCW to better reflect what this
really is
- remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas
- factor out the interpretation of SATA signatures in sata_interpet_sig()

propagate these changes to the ATA HBA drivers, and add support for PMP
to ahcisata(4) and siisata(4).

Thanks to:
- Protocase (http://www.protocase.com/) which provided a system
with lots of controllers, SATA PMP and drive slots
- Conservation Genomics Laboratory, Department of Biology, New Mexico State
University for hosting the above system
- Brook Milligan, who set up remote access and has been very responsive
when SATA cable move was needed
 1.393 26-Jun-2012  bouyer In some case, when an error is reported by the disk, the ahci controller
still reports a number of bytes transfered equal to bcount.
This then triggers a KASSERT in physio_biodone:
if (done == todo)
KASSERT(bp->b_error == 0);
Detect this case in wd(4) (so that the workaround works for other controllers
too if they have the same issue, or if the issue is with the drive)
and claim we didn't read/write anything.
 1.392 02-Feb-2012  tls branches: 1.392.2;
Entropy-pool implementation move and cleanup.

1) Move core entropy-pool code and source/sink/sample management code
to sys/kern from sys/dev.

2) Remove use of NRND as test for presence of entropy-pool code throughout
source tree.

3) Remove use of RND_ENABLED in device drivers as microoptimization to
avoid expensive operations on disabled entropy sources; make the
rnd_add calls do this directly so all callers benefit.

4) Fix bug in recent rnd_add_data()/rnd_add_uint32() changes that might
have lead to slight entropy overestimation for some sources.

5) Add new source types for environmental sensors, power sensors, VM
system events, and skew between clocks, with a sample implementation
for each.

ok releng to go in before the branch due to the difficulty of later
pullup (widespread #ifdef removal and moved files). Tested with release
builds on amd64 and evbarm and live testing on amd64.
 1.391 24-Jan-2012  jakllsch Rework struct ata_command to support LBA28 and LBA48-protocol commands.
Implement 28- and 48-bit command issuance and parameter read-back in the
various ATA host drivers. Add LBA28-protocol support to ATAIOCCOMMAND ioctl.
 1.390 25-Nov-2011  joerg branches: 1.390.2;
Replace broken memset with M_ZERO
 1.389 27-Oct-2011  jakllsch branches: 1.389.2;
Attempt to complete the WD_QUIRK_SPLIT_MOD15_WRITE list.
Per Seagate Publication number 100221381, Rev. B, the ST360015AS
is the only other drive in the Seagate Barracuda Serial ATA V
family that was not already listed.

Also, correct spelling of "globing".
 1.388 05-Oct-2011  jakllsch Limit wd(4) transfers to 128 (512-byte) logical sectors, as the traditional
MAXPHYS value has for at least the past decade.

There are issues in wd(4) and possibly in host adapter drivers that need
work before this should be raised.
 1.387 30-Jul-2011  jakllsch Add some additional consistency checking paranoia with KASSERT.
 1.386 10-Feb-2011  enami Start new line before issuing IDENTIFY command since messages from
other driver may interfere during waiting for command completion.
 1.385 05-Nov-2010  dyoung branches: 1.385.2; 1.385.4;
Mark a buffer with EIO in wdstrategy() if the device is not "enabled"
(more like "present"), not if it is merely asleep.

If the device is not awake, get out of wdstart() right away.

Brett Lymn and KAMADA Ken'ichi report that this stops the kernel from
crashing after a suspend/resume cycle.
 1.384 24-Feb-2010  dyoung branches: 1.384.2;
A pointer typedef entails trading too much flexibility to declare const
and non-const types, and the kernel uses both const and non-const
PMF qualifiers and device suspensors, so change the pmf_qual_t and
device_suspensor_t typedefs from "pointers to const" to non-pointer,
non-const types.
 1.383 23-Jan-2010  bouyer branches: 1.383.2;
struct buf::b_iodone is not called at splbio() any more.
Make sure non-MPsafe iodone callbacks raise the SPL as appropriate.
 1.382 19-Jan-2010  pooka Get rid of last "easy" kernel symbols starting with __:
__assert -> kern_assert
__sigtimedwait1 -> sigtimedwait1
__wdstart -> wdstart1

The rest are MD and/or shared with userspace, so they will require
a little more involvement than what is available for this quick
"ride the 5.99.24 bump" action.
 1.381 08-Jan-2010  dyoung Expand PMF_FN_* macros.
 1.380 17-Dec-2009  bouyer ATA sepcs changed the LBA48 boundary from 0xfffffff to 0xffffffe between
ATA6r1 and ATA6r3, which caused drives conform to ATA6r3 or later to
reject LBA28 commands at sector 0xfffffff.
Get the right idea from the LBA48 boundary from IDENTIFY words 60-61.
Remove the WD_QUIRK_FORCE_LBA48 quirk, associated tables entries and
autodetect code, it's not needed any more.
Based on patch sent to teck-kern by Christoph Badura, use of words 60-61
instead of a constant for the LBA48 boundary by me.
 1.379 19-Oct-2009  bouyer Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !
 1.378 08-Jun-2009  jakllsch Add atap_max_lba member to ataparams for LBA48 Maximum Address. From OpenBSD.
While here, renumber some __reserved member names to be sequential.
Also, add a member for the World Wide Name.

Use atap_max_lba in wd.c instead of offsets into a __reserved member.
 1.377 20-May-2009  dyoung On second thought, let's call disk_predetach() disk_begindetach().
Verbs are good.
 1.376 19-May-2009  dyoung Use disk_predetach().
 1.375 19-May-2009  dyoung Delete wdactivate() and sdactivate(). They were no-ops, but as a
side-effect of registering them, config_detach(9) cleared DVF_ACTIVE
before it called wddetach() or sddetach(). Even though sd(4)'s
detachment may have subsequently failed with EBUSY, we could not
begin new disk I/O on sd(4) because the device had been deactivated.

By analogy to sdstrategy(), test device_is_active() in wdstrategy()
and if it is false set b_errno to EIO instead of initiating new
disk I/O.

XXX We should decline more politely to start new I/O, since
XXX !device_is_active() may mean simply that the device is suspended.
XXX I suppose that EIO is safe as long as system suspension is
XXX all-or-nothing.
 1.374 15-May-2009  dyoung Do not detach a wd(4) instance if it is open, but return EBUSY,
instead. If the detach is forced, do what wdclose() does on the
last close: flush the cache, clear the flag WDF_LOADED, and delete
our reference to the ATA bus device.
 1.373 12-May-2009  cegger struct device * -> device_t, no functional changes intended.
 1.372 12-May-2009  cegger struct cfdata * -> cfdata_t, no functional changes intended.
 1.371 02-Apr-2009  dyoung During shutdown, detach devices in an orderly fashion.

Call the detach routine for every device in the device tree, starting
with the leaves and moving toward the root, expecting that each
(pseudo-)device driver will use the opportunity to gracefully commit
outstandings transactions to the underlying (pseudo-)device and to
relinquish control of the hardware to the system BIOS.

Detaching devices is not suitable for every shutdown: in an emergency,
or if the system state is inconsistent, we should resort to a fast,
simple shutdown that uses only the pmf(9) shutdown hooks and the
(deprecated) shutdownhooks. For now, if the flag RB_NOSYNC is set in
boothowto, opt for the fast, simple shutdown.

Add a device flag, DVF_DETACH_SHUTDOWN, that indicates by its presence
that it is safe to detach a device during shutdown. Introduce macros
CFATTACH_DECL3() and CFATTACH_DECL3_NEW() for creating autoconf
attachments with default device flags. Add DVF_DETACH_SHUTDOWN
to configuration attachments for atabus(4), atw(4) at cardbus(4),
cardbus(4), cardslot(4), com(4) at isa(4), elanpar(4), elanpex(4),
elansc(4), gpio(4), npx(4) at isa(4), nsphyter(4), pci(4), pcib(4),
pcmcia(4), ppb(4), sip(4), wd(4), and wdc(4) at isa(4).

Add a device-detachment "reason" flag, DETACH_SHUTDOWN, that tells the
autoconf code and a device driver that the reason for detachment is
system shutdown.

Add a sysctl, kern.detachall, that tells the system to try to detach
every device at shutdown, regardless of any device's DVF_DETACH_SHUTDOWN
flag. The default for kern.detachall is 0. SET IT TO 1, PLEASE, TO
HELP TEST AND DEBUG DEVICE DETACHMENT AT SHUTDOWN.

This is a work in progress. In future work, I aim to treat
pseudo-devices more thoroughly, and to gracefully tear down a stack of
(pseudo-)disk drivers and filesystems, including cgd(4), vnd(4), and
raid(4) instances at shutdown.

Also commit some changes that are not easily untangled from the rest:

(1) begin to simplify device_t locking: rename struct pmf_private to
device_lock, and incorporate device_lock into struct device.

(2) #include <sys/device.h> in sys/pmf.h in order to get some
definitions that it needs. Stop unnecessarily #including <sys/device.h>
in sys/arch/x86/include/pic.h to keep the amd64, xen, and i386 releases
building.
 1.370 10-Feb-2009  tron Backout LBA 48 quick entries which were added to fix one aspect of
PR kern/40569 because of objections by Manual Bouyer.
 1.369 09-Feb-2009  tron Add two more entries to the quirk table for hard disks which need the
LBA 48 work around. The first entry will watch the Seagate ST3160815AS
(and similar models), the second one HP's OEM version of the same drive.

This avoids the RAID rebuild problems described in PR kern/40569.
 1.368 06-Feb-2009  drochner branches: 1.368.2;
-in the wdc@pcmcia driver, don't try to powerup/down while cold or
shutting down -- this is only supposed to happen from the pcmcia
kernel thread while the system is running
-in wd_shutdown, call ata_addref first, so that the adapter is either
active or we know that it isn't
This fixes a hang on shutdown-p reported by Dennis den Brok per
PR kern/40531. Verified by by Dennis and me.
(reported against 5.0-Beta, so this is a pullup candidate)
 1.367 13-Jan-2009  yamt g/c BUFQ_FOO() macros and use bufq_foo() directly.
 1.366 16-Dec-2008  christos replace bitmask_snprintf(9) with snprintb(3)
 1.365 13-Dec-2008  christos more debugging in the dump path.
 1.364 05-Dec-2008  dyoung In wddetach(), avoid a crash by destroying the callout sc_restart_ch.
 1.363 08-Jun-2008  tsutsui branches: 1.363.4; 1.363.6; 1.363.8;
Use device_lookup_private() rathter than device_private(device_lookup()).
 1.362 02-May-2008  plunky branches: 1.362.2;
disk_destroy() is needed when the disk structure is no longer required,
this fixes PR kern/38131
 1.361 28-Apr-2008  martin Remove clause 3 and 4 from TNF licenses
 1.360 18-Mar-2008  cube branches: 1.360.2; 1.360.4;
Split device_t and softc for ATA devices, as well as wd(4). Other
cosmetic changes where appropriate.
 1.359 29-Feb-2008  bouyer The Hitachi HDP725025GLA380 returns "aborted command" instead of
"id not found" when hitting the LBA48 bug, so also install the
LBA48 bug workaround on "aborted command" errors.
 1.358 29-Feb-2008  dyoung Use PMF_FN_ARGS, PMF_FN_PROTO.
 1.357 28-Feb-2008  drochner use pmf again to register the shutdown handler, instead of
shutdownhook_establish()
 1.356 21-Feb-2008  drochner put back the old shutdown handlers to flush the disk cache,
the pmf API can't deal with all the different suspend/resume/reboot
cases well yet, so better keep suspend/resume and reboot/halt/poweroff
clearly seperated
 1.355 02-Jan-2008  ad branches: 1.355.2; 1.355.6;
Merge vmlocking2 to head.
 1.354 18-Dec-2007  joerg Restore suspend handler. Put the disc into standby on suspend and for
powerdown, but not for normal halt or reboot or panic. Fixes PR 37508.
 1.353 11-Dec-2007  jmcneill Make this compile again.
 1.352 11-Dec-2007  jmcneill Disable the wd_suspend power handler on the proper branch, workaround for
kern/37508
 1.351 09-Dec-2007  jmcneill branches: 1.351.2;
Merge jmcneill-pm branch.
 1.350 07-Nov-2007  itohy branches: 1.350.2; 1.350.4;
more KNF changes
 1.349 07-Nov-2007  itohy KNF or make it compile by gcc 2
 1.348 31-Oct-2007  jnemeth branches: 1.348.2;
PR/37251 - Brian Buhrow -- FORCE_LBA48 on seagate drives larger then 1TB
 1.347 19-Oct-2007  ad machine/{bus,cpu,intr}.h -> sys/{bus,cpu,intr}.h
 1.346 08-Oct-2007  ad branches: 1.346.2;
Merge disk init changes from the vmlocking branch. These seperate init /
destroy of 'struct disk' from attach / detach.
 1.345 16-Sep-2007  bouyer branches: 1.345.2;
Add a workaround for drives with the LBA48 bug:
if we get a "ID not found" error for a transfer crossing LBA48_THRESHOLD,
and the drive is larger than 128GB, automatically add WD_QUIRK_FORCE_LBA48
to the drive's quirks and retry the transfers.
Hopefully this will obsolete the WD_QUIRK_FORCE_LBA48 quirk list ...
 1.344 05-Sep-2007  jnemeth yet more caddr_t fallout
 1.343 30-Jul-2007  taca branches: 1.343.4; 1.343.6; 1.343.8;
Make it compile with WD_SOFTBADSECT option.
 1.342 29-Jul-2007  ad It's not a good idea for device drivers to modify b_flags, as they don't
need to understand the locking around that field. Instead of setting
B_ERROR, set b_error instead. b_error is 'owned' by whoever completes
the I/O request.
 1.341 21-Jul-2007  ad Replace some uses of lockmgr().
 1.340 09-Jul-2007  ad branches: 1.340.2;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements
 1.339 01-Jul-2007  dsl Allow callers of the ATAIOCCOMMAND ioctl to request the WDCC_IDENTIFY to
specify a buffer that is less than 512 bytes.
 1.338 08-Apr-2007  dyoung Include opt_ata.h for ATADEBUG definition instead of #defining it
unconditionally.
 1.337 04-Mar-2007  christos branches: 1.337.2; 1.337.4;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.
 1.336 09-Feb-2007  ad branches: 1.336.2;
Merge newlock2 to head.
 1.335 16-Nov-2006  christos branches: 1.335.2; 1.335.4;
__unused removal on arguments; approved by core.
 1.334 09-Nov-2006  bouyer Yet another broken seagate drive.
 1.333 26-Oct-2006  thorpej Use the new prop_dictionary_util functions.
 1.332 25-Oct-2006  thorpej - Add a new disk ioctl (DIOCGDISKINFO) to get the disk-info dictionary
for the disk.
- Add a new function, disk_ioctl(), that does generic disk ioctl handling.
DIOCGDISKINFO is handled here now, and others will be added in the future.
- In the wd driver, fill in the dk_info member of struct disk and use the
new disk_ioctl() function.
 1.331 12-Oct-2006  christos - sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386
 1.330 22-Sep-2006  thorpej - Define disk information, disk geometry, and disk partition dictionary
schemas. Disk information and disk geometry are designed to replace
information currently conveyed to user space using struct disklabel.
- Add a dk_info member to struct disk; a reference to a disk information
dictionary. This dictionary is to be allocated and the reference stored
in struct disk by individual drivers.
- disk_detach0() will release dk_info if non-NULL.
- Convert the wd(4) driver to stash geometry and other disk properties
as the "disk-info" property in its properties dictionary. This needs
some cleanup, but will serve as an example of what to do with other
disk drivers.
 1.329 27-Aug-2006  christos branches: 1.329.2; 1.329.4;
Oops, this is not ready yet.
 1.328 27-Aug-2006  christos Add missing initializer
 1.327 01-Aug-2006  lukem If there's any wd quirks in use, display them.
 1.326 24-Jun-2006  gendalia Add yet another broken 160GB seagate drive. Mine. ST3160021A.
 1.325 05-Jun-2006  bouyer branches: 1.325.2;
Adjust quirk entry for seagate drives. Should fix kern/33637 by Shigeya
Suzuki.
 1.324 18-May-2006  bouyer branches: 1.324.2;
Add yet another broken 160Gb seagate drive. I though we had found all of
them ... reported by Damian Lubosch on current-users.
 1.323 05-Apr-2006  cherry Backing out previous commit for more review.
 1.322 04-Apr-2006  cherry Sleeps hard disk just before system shutdown.
Some hardware (eg: Apple iBooks) don't do this automatically
and the harddisk behaves as if the power plug were pulled.

Closes PR: kern/32766
 1.321 28-Mar-2006  thorpej Use device_unit().
 1.320 01-Mar-2006  yamt branches: 1.320.2; 1.320.4; 1.320.6;
merge yamt-uio_vmspace branch.

- use vmspace rather than proc or lwp where appropriate.
the latter is more natural to specify an address space.
(and less likely to be abused for random purposes.)
- fix a swdmover race.
 1.319 20-Feb-2006  thorpej Use device_is_active() rather than testing dv_flags for DVF_ACTIVE
directly.
 1.318 15-Jan-2006  abs branches: 1.318.2; 1.318.4;
cast 16bit atap_capacity to u_int64_t before << 16. (assigned to
a u_int64_t variable). Fixes 1 to 2 TB disks in qemu. Tested with
1200GB and to a lesser extent 2047GB sparse files. For reference
an empty FFS1 filesystem on a 1200GB sparse file uses 19GB of disk.
 1.317 07-Jan-2006  yamt remove B_EINTR as it isn't used anymore.
 1.316 04-Jan-2006  yamt - add simple functions to allocate/free a buffer for i/o.
- make bufpool static.
 1.315 26-Dec-2005  yamt branches: 1.315.2;
- add ioctls to set/get disk bufq strategy.
- implement them for some drivers.
 1.314 11-Dec-2005  christos merge ktrace-lwp.
 1.313 01-Nov-2005  martin Rework the funny "user_strat: No ioctl" error messages - the functions
have not been called user_strat for quite some time and the message is
confusing if you do not know the code.
 1.312 15-Oct-2005  yamt branches: 1.312.2;
- change the way to specify a bufq strategy. (by string rather than by number)
- rather than embedding bufq_state in driver softc,
have a pointer to the former.
- move bufq related functions from kern/subr_disk.c to kern/subr_bufq.c.
- rename method to strategy for consistency.
- move some definitions which don't need to be exposed to the rest of kernel
from sys/bufq.h to sys/bufq_impl.h.
(is it better to move it to kern/ or somewhere?)
- fix some obvious breakage in dev/qbus/ts.c. (not tested)
 1.311 05-Sep-2005  riz Fix typo in comment from previous revision.
 1.310 05-Sep-2005  bouyer Use a pmatch(9) expression which should catch all present and future
seagate drives larger than 200GB for the WD_QUIRK_FORCE_LBA48 quirks.
Based on infos from
http://www.seagate.com/support/kb/disc/howto/interpret_model.html
 1.309 05-Sep-2005  bouyer Add quirk for ST3200826A. From rudolf (netbsd at eq.cz) on tech-kern.
 1.308 29-Aug-2005  bouyer wd->atabus->ata_bio() should handle the multi-sector PIO transfers just fine,
and in a better way than what is done in wddump() (which also does it
for DMA transfers, limiting the dump speed significantly). So remove
code splitting transfers in wd->sc_multi chunks from wddump(), and call
wd->atabus->ata_bio() with the whole transfers.

Problem reported by Frank Kardel, and patch tested by him.
 1.307 26-Aug-2005  briggs ST3300831A also needs WD_QUIRK_FORCE_LBA48.
 1.306 25-Aug-2005  bouyer Add a comment explaining why we have WD_QUIRK_FORCE_LBA48, instead of always
using LBA48 for sector 0xffffff.
 1.305 02-Jul-2005  dsainty branches: 1.305.2;
Force LBA48 access to ST3160827AS drives. Without this change the entire
system hangs whenever a read or write request to sector 0xfffffff occurs.

The hang appears to happen when:

status = bus_space_read_1(wdr->cmd_iot, wdr->cmd_iohs[wd_status], 0);

is executed in __wdcwait(), and it is not possible to enter the debugger or
make the system respond at all after this point.

Whilst not necessarily the most prudent solution to handling the large number
of drives that exhibit this problem, it is currently the popular workaround.

[An alternative approach is to just always drop to LBA48 if the request
happens to pass by sector 0xfffffff and the drive reported as larger than
0xfffffff sectors. My understanding is that the 32 bit addressing below
0x10000000 is purely there as a performance booster, not to resolve a
compatibility issue.]
 1.304 19-Jun-2005  bouyer The ST3160023A* also need the LBA48 quirk. From Timo Schoeler.
 1.303 16-Jun-2005  bouyer Use pmatch() metacharacters to match both SATA and PATA versions of seagate
drives in the quirk table.
 1.302 16-Jun-2005  martti Added LBA48 quirk for Seagate ST3200822AS. Tested by Teemu Rinta-aho.
 1.301 29-May-2005  christos - remove casts [to avoid volatile]
- sprinkle const
- avoid shadowed variables
 1.300 16-May-2005  bouyer Add another broken seagate device to the quirk list.
Problem reported by rudolf <netbsd at eq.cz> on netbsd-help@
 1.299 31-Mar-2005  yamt introduce a function to drain bufq and use it where appropriate.
 1.298 27-Feb-2005  perry branches: 1.298.2;
nuke trailing whitespace
 1.297 18-Nov-2004  bouyer branches: 1.297.4; 1.297.6;
Some drives reports ATA version < 4, and still support (and need) the
WDCC_FLUSHCACHE command. So for drive < ATA4, test WDC_CMD2_FC in atap_cmd_set2
before returning ENODEV.
Problem reported and fix tested by Michael Gerhards on port-sparc64.
 1.296 28-Oct-2004  yamt move buffer queue related stuffs from buf.h to their own header, bufq.h.
 1.295 28-Sep-2004  bouyer Add a WD_QUIRK_FORCE_LBA48 quirck, which forces all access to use LBA48.
Use it for ST3200822A, which has a broken firmware and return an error
when accessing sector 0xfffffff with a LBA command.
Patch tested by David Brownlee.
 1.294 25-Sep-2004  thorpej Add support for wedges to the ATA disk driver.
 1.293 16-Sep-2004  bouyer In case of error, reset the channel with AT_RST_NOCMD. We know the reset
won't abort any command at this point.
 1.292 30-Aug-2004  drochner make the software managed bad-sector list optional
(kernel option WD_SOFTBADSECT)
 1.291 21-Aug-2004  thorpej atastart() (called only at splbio(), and from interrupts) can change
drive_flags, to make sure all drive_flags manipulations are done at
splbio().
 1.290 20-Aug-2004  thorpej Rename the ata_bustype "ata_reset_channel" member to "ata_reset_drive"
to reflect the new reality after Manuel added the bus reset ioctls.
 1.289 13-Aug-2004  thorpej WDCDEBUG -> ATADEBUG.
 1.288 12-Aug-2004  thorpej Rename some constants:
WDC_COMPLETE -> ATACMD_COMPLETE
WDC_QUEUED -> ATACMD_QUEUED
WDC_TRY_AGAIN -> ATACMD_TRY_AGAIN
 1.287 12-Aug-2004  thorpej Rename "struct wdc_command" to "struct ata_command".
 1.286 10-Aug-2004  mycroft A few things:
* Kill off device nodes *before* we clear out the queue, to help prevent any
new I/O being queued.
* More useful error reporting in wd_setcache() and wd_flushcache().
* Add a wd_standby() (tested but not used yet).

And the most important:
* In wddetach(), if the device was open, call adapt_delref() so that we
propagate the disable up to our PCMCIA socket.
 1.285 04-Aug-2004  bouyer Change wdc_kill_pending() to take a struct ata_drive_datas * as argument,
and kill only pending requests for this drive.
Implement a DRIVE_WAITDRAIN flag, which will cause the active command to
be killed once complete.
Other minor fixes.
Now it's possible to detach a ATA or ATAPI device from ioctl even when
a dd on the raw char partition is running.
 1.284 03-Aug-2004  bouyer First cut at implementing ATABUSIODETACH. Still buggy.
 1.283 03-Aug-2004  bouyer wddetach(): move lockmgr(LK_DRAIN) near the end; otherwise we'll panic in
wddone() (vdevgone() may end up calling it) trying to get the lock.
wdopen(): check for DVF_ACTIVE.
 1.282 03-Aug-2004  bouyer move cleaning of the bad block list in wddetach() after the call to
disk_detach(), to avoid a race condition with the interrupt code.
 1.281 31-Jul-2004  bouyer Implement asynchronous channel reset.
Use this to reset the channel before doing a dump, instead of the hack in
wdc_exec_xfer() based on C_POLL. This hack was causing problems on
controllers with a shared queue, because we now can have C_POLL set during
concurent channels probes (problem found and analysed on sparc64 by
Martin Husemann).
This should even make core dumps marginally more reliable on ATA drives.
 1.280 22-Jun-2004  mycroft Update copyright.
 1.279 22-Jun-2004  mycroft Do not force ATA_SINGLE on devices that don't support READ/WRITE MULTI. We
just have to take an interrupt for each sector.
Tested with one laptop disk (which normally runs in DMA mode and was forced
to single-sector transfers) and 3 CF cards. Increases the performance of
the CF cards substantially (760KB/s->1240KB/s in one case, 410KB/s->750KB/s
in the other two cases).
 1.278 01-Jun-2004  mycroft Fix an extremely obvious bug in the handling of the bad block list: the "max"
block was being set 512x further out than it should be, causing rather severe
escalation of the error.

XXX WTF is the point of this shit, anyway? In most cases, the way you're
supposed to fix a bad block on an ATA disk is to rewrite it -- which will
either just transparently fix it, or spare it. This code actively prevents
that.
 1.277 27-May-2004  thorpej Clean-up of use of "precomp". It is used as "features" everywhere, so
rename the fields in wdc_command as appropriate.
 1.276 21-May-2004  bouyer Add support for the DIOCCACHESYNC ioctl.
 1.275 03-May-2004  bouyer When using LBA48, use Flush cache extended command if supported.
From Christos Zoulas in PR 25366, with minor adjustements from me.
 1.274 28-Feb-2004  yamt branches: 1.274.2;
change the way to handle NEW_BUFQ_STRATEGY option.
instead of putting #ifdefs into each drivers,
use a global variable to indicate default strategy.

XXX should have a way to specify other strategies.
 1.273 10-Jan-2004  yamt store a i/o priority hint in struct buf for buffer queue discipline.
 1.272 14-Dec-2003  thorpej Move the wddone() prototype to wd.c
 1.271 14-Dec-2003  thorpej Garbage-collect __ATA_DISK_PRIVATE; it is no longer needed.
 1.270 14-Dec-2003  thorpej Instead of calling wddone() directly, set a (*drv_done)() callback in
the ata_drive_datas structure and invoke that.
 1.269 04-Dec-2003  keihan netbsd.org -> NetBSD.org

This was the last commit of this kind to src/sys, which is now totally
"NetBSD.org clean". Thanks for the patiance, and sorry for all the commits.
 1.268 30-Nov-2003  yamt - rename WDC_CAP_LBA48 to ATA_CMD2_LBA48.
- remove WDC_CAP_QUEUE. (there's already ATA_CMD2_RWQ.)

as discussed on tech-kern@.
 1.267 07-Nov-2003  mycroft Replace home-grown locks with lockmgr().
 1.266 29-Oct-2003  mycroft Back out the bogus initializer -- the compiler bug is fixed.
 1.265 28-Oct-2003  cl note 'm68k {u,}int64_t used uninitialized' bug.
add reference to gcc bug report.
mark all (known) occurrences.
 1.264 27-Oct-2003  fredb Appease gcc3 "-Wno-uninitialized".
 1.263 08-Oct-2003  bouyer Following Matt Thomas's request, rename ata attribute to ata_hl, and
wdc_base to ata. We can now have
atabus* at ata?
in kernel config files.
 1.262 08-Oct-2003  bouyer Make the ATA mid-layer appears as atabus, as proposed in
http://mail-index.netbsd.org/tech-kern/2003/09/25/0006.html
This adds a device (atabus) between IDE controllers and wd or atapibus, to
have each ATA channel show up in the device tree. Later there will be atabus
devices in /dev, so that we can do IOCTL on them.
Each atabus has its own kernel thread, to handle operations that needs polling,
e.g. reset and others.

Device probing on each bus it defered to the atabus thread creation.
This allows to do the reset and basic device probes in parallel, which reduce
boot time on systems with several pciide controllers.
 1.261 19-Sep-2003  mycroft 1) Use config_interrupts() to attach IDE and ATAPI drives. This eliminates
most polling.
2) Clean up some goofiness in pciide -- get rid of the whole "candisable" path
(it's gratuitous) and simplify the code by calling pciide_map_compat_intr(),
*_set_modes() and wdc_print_modes() from central locations.
3) Add a register writability and register ghost test to eliminate phantom
drives more quickly.
 1.260 03-Aug-2003  bouyer For LBA drives, compute the number of cylinders from the drive capacity
in the default disklabel and the boot message, instead of using the
value reported by the drive (which is 16383 if the drive is larger than 8G).
Should fix PR 9864
 1.259 11-Jul-2003  dsl KNF and whitespace (and indent a contuation line)
 1.258 10-Jul-2003  matt Add LBA48_THRESHOLD #define and set it to the *correct* value. Use it
in place of "0xffffff" (which is one too few 'f').
 1.257 29-Jun-2003  fvdl branches: 1.257.2;
Back out the lwp/ktrace changes. They contained a lot of colateral damage,
and need to be examined and discussed more.
 1.256 29-Jun-2003  bouyer Don't use LBA48 if we don't have to (that is, if the request is inside the
first 128GB). Tested by Andreas Johansson.
 1.255 29-Jun-2003  bouyer Remove unused WAITTIME definition. Pointed out by Andreas Johansson.
 1.254 28-Jun-2003  darrenr Pass lwp pointers throughtout the kernel, as required, so that the lwpid can
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.

Bump the kernel rev up to 1.6V
 1.253 17-May-2003  thorpej * Use aprint*().
* Add Ultra/133 to wdc_print_modes().
 1.252 10-May-2003  thorpej Change bounds_check_with_label() to take a pointer to the disk structure,
rather than the label itself. This paves the way for some future changes.
 1.251 02-May-2003  dsl Change return type of readdisklabel() to const char *
I hope I've found all the correct places!
 1.250 27-Apr-2003  bouyer Fix typo.
 1.249 27-Apr-2003  bouyer Acceptable cache operations are r and rw, not none and w
Properly initialise the timeout value.
 1.248 26-Apr-2003  bouyer Add support for the getcache/setcache ioctls.
 1.247 18-Apr-2003  explorer add my 80 GB SATA drive to the quirk table.
 1.246 18-Apr-2003  mason From current-users:

On Thu, Apr 17, 2003 at 11:26:26AM -0500, Dave Huang wrote:

> Typo police here :) "whose," not "who's."
 1.245 16-Apr-2003  darrenr add missing setting of b_error (thanks Manuel)
 1.244 15-Apr-2003  nathanw Add a missing paren.
 1.243 15-Apr-2003  darrenr Implement (and document!) "dkctl <dev> badsector .." to `manage' the bad
sector information in the kernel. Doing this uncovered some shortcomings
that should have been pretty obvious with the code committed prior, addressing
the major kludge with a new struct - disk_bacsecinfo to be passed into
DIOCBSLIST.
 1.242 15-Apr-2003  dogcow fix typos
 1.241 15-Apr-2003  darrenr Automatically build up a list of bad sectors from failed read operations and
prempt read operations on matching regions with a failure rather than waiting
for the device to return a failure. The I/O operation must have already failed
MAXRETRIES times before being added to the list - this can generally take up
to 12 seconds.
List is made accessible to userspace via DIOCBSLIST and DIOCBSFLUSH.
 1.240 03-Apr-2003  fvdl Check RAW_PART against the media size instead of the disklabel.
Add the media size in 512-byte sectors to the softc, to avoid
some 64 bit computations. Bump the capacity stored in softcs
for disks to 64 bits.
 1.239 21-Mar-2003  thorpej Work around a buggy PHY in some Seagate S-ATA drives which get confused
by the way some S-ATA controllers packetize data when the sector count of
the transfer mod 15 is 1.
 1.238 23-Feb-2003  yamt initialize b_interlock properly.
(for ioctl)
 1.237 27-Jan-2003  thorpej Experimental support for RAID volumes configured by ATA "RAID" controllers.
Such RAID controllers are actually just IDE controllers with a BIOS that
can create RAID volumes and write the configuration info to config blocks
on the disks. The BIOS can do I/O to these volumes, and the OS must
understand the config blocks and implement RAID in software in order to be
able to use these volumes.

Only SPAN (simple concatenation) and RAID0 are supported at this time,
and writing back config blocks is also not supported at this time. Currently,
only the Promise configuration scheme is supported, although supporting
the Highpoint scheme should not be too difficult.

In any case, this is sufficient to use the Promise RAID0 volume (thus
preserving the win2k AS installation) on this new Intel server I have.

Thanks to Soren Schmidt for doing the work in FreeBSD; it made this
task much easier. The config block parsing code is adapted from his
work.
 1.236 23-Jan-2003  bad Being able to make image backups of your whole disk, not to mention not
causing certain ata disks to lock up by reading beyond the last block,
beats catering to broken devices.

bounds_check_with_label() RAW_PART too.
 1.235 20-Jan-2003  simonb The Double-Semi-Colon Police.
 1.234 07-Jan-2003  fvdl Fix error in ODIOCGDINFO handling caused by previous change to it.
 1.233 15-Dec-2002  fvdl Avoid using a lot of kernel stack.
 1.232 01-Nov-2002  mrg implement separate read/write disk statistics:
- disk_unbusy() gets a new parameter to tell the IO direction.
- struct disk_sysctl gets 4 new members for read/write bytes/transfers.
when processing hw.diskstats, add the read&write bytes/transfers for
the old combined stats to attempt to keep backwards compatibility.

unfortunately, due to multiple bugs, this will cause new kernels and old
vmstat/iostat/systat programs to fail. however, the next time this is
change it will not fail again.

this is just the kernel portion.
 1.231 23-Oct-2002  jdolecek merge kqueue branch into -current

kqueue provides a stateful and efficient event notification framework
currently supported events include socket, file, directory, fifo,
pipe, tty and device changes, and monitoring of processes and signals

kqueue is supported by all writable filesystems in NetBSD tree
(with exception of Coda) and all device drivers supporting poll(2)

based on work done by Jonathan Lemon for FreeBSD
initial NetBSD port done by Luke Mewburn and Jason Thorpe
 1.230 18-Oct-2002  junyoung Add option NEW_BUFQ_STRATEGY, a new buffer queue strategy for better
system responsiness under high disk I/O load.

OK'ed by Juergen Hannken-Illjes.
 1.229 02-Oct-2002  thorpej Add trailing ; to CFATTACH_DECL.
 1.228 30-Sep-2002  thorpej Use CFATTACH_DECL().
 1.227 27-Sep-2002  thorpej Declare all cfattach structures const.
 1.226 27-Sep-2002  drochner Remove a check whether the drive is initialized before crashdump.
It prevents dumping on an otherwise unused disk and seems
unnecessary anyway - the driver will deal with initialization.
fixes my PR kern/18135
 1.225 06-Sep-2002  gehenna Merge the gehenna-devsw branch into the trunk.

This merge changes the device switch tables from static array to
dynamically generated by config(8).

- All device switches is defined as a constant structure in device drivers.

- The new grammer ``device-major'' is introduced to ``files''.

device-major <prefix> char <num> [block <num>] [<rules>]

- All device major numbers must be listed up in port dependent majors.<arch>
by using this grammer.

- Added the new naming convention.
The name of the device switch must be <prefix>_[bc]devsw for auto-generation
of device switch tables.

- The backward compatibility of loading block/character device
switch by LKM framework is broken. This is necessary to convert
from block/character device major to device name in runtime and vice versa.

- The restriction to assign device major by LKM is completely removed.
We don't need to reserve LKM entries for dynamic loading of device switch.

- In compile time, device major numbers list is packed into the kernel and
the LKM framework will refer it to assign device major number dynamically.
 1.224 21-Jul-2002  hannken Rename bufq_init() to bufq_alloc().
Add bufq_free() to remove a buffer queue.
Avoid MALLOC while holding a spinlock.

From Chuck Silvers.
 1.223 16-Jul-2002  hannken Implement a new device buffer queue interface.
One basic struct, a function to setup a queue with a specific strategy and
three macros to put buf's into the queue, get and remove the next buf or
get the next buf without removal.

The BUFQ_XXX interface will be removed in the future.
The B_ORDERED flag is not longer supported.

Approved by: Jason R. Thorpe <thorpej@wasabisystems.com>
 1.222 28-Jun-2002  yamt do previous differently for wddump case.
 1.221 28-Jun-2002  yamt - remove big buffers on stack. discussed on tech-kern.
- constify.
 1.220 13-Jan-2002  christos branches: 1.220.8; 1.220.10;
Add LBA48 support based on patches from Keisuke YOSHIDA. KNF a bit while
I am there.
 1.219 12-Jan-2002  tsutsui Call malloc(9) with M_ZERO flag instead of memset() after malloc().
 1.218 03-Dec-2001  bouyer Update my copyrigth.
 1.217 03-Dec-2001  bouyer Allow wd to attach to something else than wdc/pciide (like USB<->ATA bridges):
- move some functions from ata.c to ata_wdc.c or wdc.c.
- add callbacks to struct ata_bustype so that wd.c doesn't call directly
functions from the lower level driver.
 1.216 02-Dec-2001  bouyer Clean up attach of wd/atapibus:
kill ata_atapi_attach. Change atapibus to use a struct scsipi_channel instead
of ata_atapi_attach as attach arch. Create a ata_device, compatible with
scsipi_channel, to attach wd.
 1.215 13-Nov-2001  lukem add RCSID
 1.214 13-Jun-2001  bjh21 branches: 1.214.2; 1.214.4;
Add explicit support for IDE and SCSI adaptors which don't support interrupts.
On such adaptors, all transfers are done in polling mode.

OK'ed by Manuel on tech-kern.
 1.213 06-May-2001  drochner Since __HAVE_OLD_DISKLABEL, DISKMINOR(u, p) isn't continuous over "p"
anymore. So we can't pass a range to vdevgone() but have to loop over the
partitions explicitely.
(otherwise the next device unit will be killed)
 1.212 08-Jan-2001  fvdl branches: 1.212.2;
Return error in the case of using ODIOCGDINFO or ODIOCGDEFLABEL when
the number of partitions is > OLDMAXPARTITIONS. This is better
than silently truncating the label (don't want to silently throw
away partitions when using an old disklabel binary on a label with
> 8 partitions). From Enami Tsugutomo.
 1.211 07-Jan-2001  fvdl Adapt all disk devices in MI directories to handle ODIOC* calls
for ports that have bumped MAXPARTITIONS (and thus define
__HAVE_OLD_DISKLABEL).
 1.210 14-Dec-2000  lukem s/pio/PIO/ and s/lba/LBA/ in messages output to users
 1.209 06-Jul-2000  thorpej Use device_lookup() *more*.
 1.208 06-Jul-2000  thorpej Use device_lookup().
 1.207 05-Jul-2000  thorpej Garbage-collect __BDEVSW_DUMP_OLD_TYPE.
 1.206 28-Jun-2000  mrg remove include of <vm/vm.h>
 1.205 27-May-2000  bouyer branches: 1.205.2;
Ops, state should be RESET, not RECAL here.
 1.204 15-May-2000  bouyer branches: 1.204.2;
Sync my copyrigth notice
 1.203 23-Mar-2000  thorpej New callout mechanism with two major improvements over the old
timeout()/untimeout() API:
- Clients supply callout handle storage, thus eliminating problems of
resource allocation.
- Insertion and removal of callouts is constant time, important as
this facility is used quite a lot in the kernel.

The old timeout()/untimeout() API has been removed from the kernel.
 1.202 07-Feb-2000  thorpej Fix a bug in disksort_*() which caused non-optimal ordering when multiple
active partitions were on a single spindle. Add a b_rawblkno member to
struct buf which contains the non-partition-relative block number to sort
by.
 1.201 24-Jan-2000  enami Use format_bytes to convert drive capacity into human familier string (like
sd.c does).
 1.200 21-Jan-2000  thorpej Update for sys/buf.h/disksort_*() changes.
 1.199 23-Dec-1999  leo Implement the DIOCKLABEL ioctl. Through this ioctl it it possible keep the
disklabel on the last close or to drop it.
 1.198 10-Nov-1999  leo Give lp->d_type a value other than 'unknown'. When the model is "ST506",
the d_type field becomes DTYPE_ST506, otherwise it will be set to DTYPE_ESDI.
 1.197 20-Oct-1999  enami Cancel active transfers on aic/wdc detach.
Also makes LS-120 drive works for me again.
 1.196 23-Sep-1999  enami branches: 1.196.2; 1.196.4; 1.196.6;
Allow to detach wdc, atapibus, wd and cd.
 1.195 22-Sep-1999  enami Fix indentation of close curly brace.
 1.194 09-Aug-1999  bouyer Don't call wddone() from _wdstart(), the ata_wdc system will do it now.
call wd_flushcache() with the proper flags (especially don't set
AT_WAIT | AT_POLL).
 1.193 28-Feb-1999  explorer branches: 1.193.2;
Update to slightly altered rnd_attach_source() api
 1.192 21-Feb-1999  hubertf Only define WDCDEBUG if not already defined (e.g. by some kernel option)
 1.191 18-Feb-1999  bouyer Add a missing ':'. pointed out by Dave Huang.
 1.190 18-Jan-1999  bouyer move wd.c:print_wderror() to ata.c:ata_perror().
In wdc_probe_caps() add code to guess the ATA revision supported (if
ATA4 if Ultra-DMA, ATA2 if PIO mode > 2). We can't rely on param.atap_ata_major
here, at last one Ultra-DMA drive claims to support only ATA-3.
Use the ATA version in ata_perror(), and to try a flush cache command
in a shutdown hook for IDE drives.
 1.189 16-Dec-1998  bouyer Switch to single-sector transfert mode only for the last retry, to let
a chance to the underlying driver to downgrade the mode first.
 1.188 23-Nov-1998  kenh Add a new flag to wdc_command: ATA_READREG. This will indicate that all
device registers should be read back into the wdc_command structure after
successfull command completion. Use this this in wdioctl() for
ATAIOCCOMMAND.
 1.187 20-Nov-1998  thorpej Implement controller reference counting on ATA devices.
 1.186 19-Nov-1998  kenh Rename WDCIOCCOMMAND (and related structures) to ATAIOCCOMMAND
 1.185 19-Nov-1998  kenh Add support for the WDCIOCCOMMAND ioctl. This allows the sending of
arbitrary ATA commands from user-space. Concepts mostly taken from
the implementation of the SCIOCCOMMAND ioctl.
 1.184 11-Nov-1998  bouyer - clearify the boot messages (features supported vs features used). Thanks to
Havard Eidnes for his complains about this :)
- fix some typo in comments
- hoppefully better detection of drives reporting bogus PIO modes.
 1.183 20-Oct-1998  bouyer Donc reset the disk state to 0 in wd.c, set a flag which will cause it
to be reset to 0 at wdcstart(). This fixes a race condition between
normal I/O and wdioclt(). Also use interrupts for wd_get_params in wdopen().
 1.182 13-Oct-1998  bouyer Properly separate DEBUF_XFERS and DEBUG_FUNCS: DEBUF_XFERS is for data
transfers (may be called often), where DEBUG_FUNCS is for setup functions
(not used for normal contitions).
 1.181 12-Oct-1998  bouyer Merge bouyer-ide
 1.180 07-Sep-1998  drochner make the default disklabel consistent to the SCSI/ATAPI case
 1.179 15-Aug-1998  rvb Use scsi style geometry printout
 1.178 15-Aug-1998  mycroft Assign my copyrights to TNF.
 1.177 13-Aug-1998  rvb Report "true" disk geometry; like sd disks do
 1.176 09-Jun-1998  thorpej Nuke __BROKEN_INDIRECT_CONFIG.
 1.175 04-Jun-1998  enami branches: 1.175.2;
Print newline like others do when tracing wdunlock.
 1.174 09-May-1998  mycroft Disable the change to check wdp_ataversion, since it's unreliable with old
devices. (Yuck.)
 1.173 26-Apr-1998  mycroft Update copyright.
 1.172 26-Apr-1998  mycroft In LBA mode, always use the `total capacity' count, rather than calculating
the capacity based on the c/h/s numbers. In fact, don't use the c/h/s
numbers for much of anything.
For ATA-4 drives or later, always use LBA mode, since it's now required.

Collectively, this allows >8GB disks (like the 12GB Bigfoot) to work.
 1.171 23-Apr-1998  bouyer From Robert V. Baron:
Compute the disk block addr at command queing time rather than exec time.
This fix a bug which could lead to data corruption on disk: when a command
was reexecuted after an error condition (from wdcunwedge), the partition
offset was re-added to the block addr, leading to a transfert at the wrong
disk block.
This should fix the problem reported by some laptop's users, where the
first disk read after a suspend/resume returned garbage.
 1.170 25-Mar-1998  leo Conditionalize badblock handling on 'HAS_BAD144_HANDLING'. Some ports
(like atari) do not support it.
 1.169 15-Jan-1998  cgd kill a couple of unnecessary #ifs
 1.168 14-Jan-1998  cgd Various cleanups and bug fixes to the wdc/wd code:
* Fix bug in wdc that would overflow ATAPI transfer length.
* Improve wdc probe code so that 'wdc' is probed in if present
even if there are no drives attached, and so that it works
properly even if the only device is an ATAPI slave.
* bus_space-ify.
* split the ISA attachment from the wdc driver, and remove
ISA dependencies from non-ISA files.
* claim that wd and wdc are now machine-independent (probably not
completely true, but mostly so; they at least work on arm32 and
i386).
* Various other minor fixups and cleanups, some of which were pointed
out by Kazuki Sakamoto.
 1.167 12-Jan-1998  thorpej Update for config changes.
 1.166 13-Oct-1997  explorer o Make usage of /dev/random dependant on
pseudo-device rnd # /dev/random and in-kernel generator
in config files.

o Add declaration to all architectures.

o Clean up copyright message in rnd.c, rnd.h, and rndpool.c to include
that this code is derived in part from Ted Tyso's linux code.
 1.165 10-Oct-1997  explorer Add hooks to insert timing info into the random system
 1.164 08-Oct-1997  thorpej Implement DIOCGDEFLABEL.
 1.163 27-Aug-1997  bouyer Merge scsipi branch in the mainline. This add support for ATAPI devices
(currently only CD-ROM drives on i386). The sys/dev/scsipi system provides 2
busses to which devices can attach (scsibus and atapibus). This needed to
change some include files and structure names in the low level scsi drivers.
 1.162 18-Jul-1997  fvdl branches: 1.162.2;
Work around possible race condition with 2 drives on one controller
in wd_get_parms. PR 3773, from Onno van der Linden (onno@simplex.nl)
 1.161 17-Jul-1997  jtk use locator defines in "locators.h" to index cf_loc[]

still to be done and/or decided not to be done: replace all *UNK uses in
isa code with the native names from locators.h
 1.160 24-Jun-1997  thorpej branches: 1.160.2;
foosize()'s return value is in DEV_BSIZE units; adjust the size obtained
from the disklabel accordingly.
 1.159 18-Jun-1997  pk Remove spurious line left over in previous commit.
 1.158 18-Jun-1997  pk wdsize(): only call wdopen()/wdclose() if not already open, per the
current `openmask'.
 1.157 06-Jun-1997  thorpej Pull thorpej-bus-dma branch into mainline.
 1.156 17-Jan-1997  perry branches: 1.156.6;
Fixes to wd driver from cgd -- corrects several nasty bugs typically
associated with coming out of suspends/sleeps on laptops.
Approved by fvdl.
 1.155 27-Nov-1996  thorpej In wdattach(), when printing out info about the drive, make the output
look a little more like a SCSI disk's. This has the pleasant side-effect
of preventing the output from annoyingly extending past 80 columns.
 1.154 07-Nov-1996  mikel Fix WDDEBUG code; PR kern/1480.
 1.153 13-Oct-1996  christos backout kprintf changes
 1.152 10-Oct-1996  christos printf -> kprintf, sprintf -> ksprintf
 1.151 27-Aug-1996  cgd change cfprint_t type definition to take a const char *, rather than
a char *, because that's what was really intended, and because
if the print function modifies the string, various things could become
unhappy (so the string should _not_ be modified).
 1.150 12-May-1996  mycroft Use intr.h.
 1.149 29-Apr-1996  christos - gcc -Wall fixes
 1.148 11-Apr-1996  cgd update for addition of a machine-dependent cookie as the first argument
to isa_intr_{,dis}establish().
 1.147 17-Mar-1996  thorpej New device attachment scheme:

- split softc size and match/attach out from cfdriver into
a new struct cfattach.

- new "attach" directive for files.*. May specify the name of
the cfattach structure, so that devices may be easily attached
to parents with different autoconfiguration semantics.
 1.146 01-Mar-1996  mycroft Add support to the ISA DMA framework for auto-initialize mode.
Add experimental SB16 code, disabled for now.
 1.145 08-Jan-1996  mycroft Make sure we disk_busy() for every transfer we start.
 1.144 07-Jan-1996  thorpej New generic disk framework. Highlights:

- New metrics handling. Metrics are now kept in the new
`struct disk'. Busy time is now stored as a timeval, and
transfer count in bytes.

- Storage for disklabels is now dynamically allocated, so that
the size of the disk structure is not machine-dependent.

- Several new functions for attaching and detaching disks, and
handling metrics calculation.

Old-style instrumentation is still supported in drivers that did it before.
However, old-style instrumentation is being deprecated, and will go away
once the userland utilities are updated for the new framework.

For usage and architectural details, see the forthcoming disk(9) manual
page.
 1.143 24-Dec-1995  mycroft The IST_* and IPL_* constants are not bus-specific; don't treat them as such.
Change splimp -> splnet in Ethernet, ARCnet, and FDDI drivers.
 1.142 05-Aug-1995  mycroft Use an intermediate variable to shorten label initialization code.
Make wddump() work on more machines.
 1.141 04-Jul-1995  mycroft Make each disk and tape driver define its own read and write functions.
Deprecate rawread() and rawwrite() completely. Remove d_strategy from cdevsw to
force the abstraction barrier.
 1.140 26-Jun-1995  cgd implement new-style block device dump routine if __BDEVSW_DUMP_OLD_TYPE
not defined, otherwise use an "not implemented" stub.
 1.139 17-Apr-1995  cgd clean up several ISA device interfaces: autoconfiguration, header
inclusion, and interrupt configuration. more work still needs to be done,
but it's getting better...
 1.138 15-Apr-1995  mycroft Don't boundary check I/O to the `raw' partition.
 1.137 10-Apr-1995  mycroft kernel_pmap --> pmap_kernel()
 1.136 01-Apr-1995  mycroft Add a missing unlock.
 1.135 29-Mar-1995  mycroft Revamp the locking mechanism slightly. Update some outdated comments, and add
a few new ones.
 1.134 25-Mar-1995  mycroft Remove the write protect check altogether, and rely on the drive to do it.
 1.133 23-Mar-1995  mycroft Check for read-only media in open(), not write().
 1.132 23-Mar-1995  mycroft Allow DIOCWDINFO even if label is not `writable'.
 1.131 27-Feb-1995  cgd use wdc_softc in wdccd, rather than wd_softc; from chuck cranor. (pr 828)
 1.130 29-Jan-1995  cgd lip service to making ISA support machine-independent. This is the first
round: moving the drivers into a machine-independent directory.
Some drivers (e.g. fd.c) not moved because they use other pc features (e.g.
CMOS settings), and none of the non-driver files moved, because they're
still pretty much PC specific. eventually (when other ports with ISA
busses really start using this code), more 'high-level' ISA support will
live here.
 1.129 13-Jan-1995  mycroft (Almost) allow sector sizes other than 512 bytes. Reduce some expensive operations.
 1.128 13-Jan-1995  mycroft Always boundary check I/O.
 1.127 13-Jan-1995  mycroft Fix another bug in the DMA code.
 1.126 13-Jan-1995  mycroft Minor rearrangement.
 1.125 13-Jan-1995  mycroft Change the way sc_skip is used. Fix a bug in the (incomplete) DMA code. Add a
new sc_blkno (which will be useful later).
 1.124 07-Jan-1995  mycroft Fix a race condition between normal I/O and wd_get_parms().
 1.123 03-Jan-1995  mycroft Add interrupt sharing types.
 1.122 14-Dec-1994  mycroft Remove dkbad.h.
 1.121 01-Dec-1994  mycroft Ignore ERR during diagnostics.
 1.120 30-Nov-1994  mycroft Clear the controller active flag after a timeout.
 1.119 23-Nov-1994  mycroft Officially credit Onno.
 1.118 23-Nov-1994  mycroft There's no point in bothering to unlock what we didn't lock.
 1.117 23-Nov-1994  mycroft Fix an outdated comment.
 1.116 23-Nov-1994  mycroft Use logical block addressing if available. Be a little more careful about
maintaining timeouts. Clean up wddump() a bit.
 1.115 22-Nov-1994  mycroft Implement 32-bit reads and writes, but for now assume 16-bit mode.
 1.114 22-Nov-1994  mycroft Add a missing newline in some error messages.
 1.113 22-Nov-1994  mycroft Implement EIDE multi-sector PIO and DMA modes, based on code from Onno van
der Linden. Also, various structural cleanup, and fix two bogons that crept
in, and remove all of the old clustering nonsense.
 1.112 22-Nov-1994  mycroft Minor fix to debugging code.
 1.111 22-Nov-1994  mycroft Fudge a couple more fields for ancient disks.
 1.110 22-Nov-1994  mycroft Change some constant names, etc., to better match the ATA spec.
 1.109 22-Nov-1994  mycroft Add still more locking.
 1.108 20-Nov-1994  mycroft Add some missing locking, and some general cleanup.
 1.107 18-Nov-1994  mycroft Convert port, IRQ, and DRQ numbers to ints.
 1.106 04-Nov-1994  mycroft Fix stupid bug in last change.
 1.105 04-Nov-1994  mycroft Add a simplified version of wdcommand() that can be used in some cases.
Avoid needing a wd_softc inside wdprobe().
 1.104 04-Nov-1994  mycroft No longer needs icu.h.
 1.103 03-Nov-1994  mycroft *intr() arg does not need to be void*.
 1.102 03-Nov-1994  mycroft Use indirect config for subdevices, and add dkdrivers where missing.
 1.101 30-Oct-1994  cgd be more careful with types, also pull in headers where necessary.
 1.100 27-Oct-1994  cgd new RCS ID format.
 1.99 20-Oct-1994  mycroft Be careful not to stack multiple timeouts.
 1.98 20-Oct-1994  mycroft Remove the local timeout mechanism, and just use explicit timeout()/untimeout()
calls.
 1.97 20-Oct-1994  mycroft Always set d_secsize to 512 for now.
 1.96 20-Oct-1994  mycroft Don't switch to single-sector mode just because we got a corrected error.
 1.95 20-Oct-1994  mycroft Rearrange wdcstart() in a more logical fashion, and eliminate many redundant
calculations. Also, speed up bad block searching a little.
 1.94 20-Oct-1994  mycroft openpart -> openmask
 1.93 20-Oct-1994  mycroft Various changes:
* Add a dkdevice; move sc_*openpart, sc_label, and sc_cpulabel into it.
* Turn sc_wlabel into a flag, and don't set it automatically in DIOCWLABEL.
* If there is no label, disallow all I/O except to the raw partition.
* Don't allow I/O to `unused' partitions.
* Beginnings of support for block sizes other than 512 bytes.
Other minor changes.
 1.92 14-Oct-1994  cgd move arch dependence to archs. clean up a bit. deal with weird MAXPARTITONS.
 1.91 09-Oct-1994  mycroft Remove unneeded #includes.
 1.90 07-Oct-1994  mycroft Don't poll the BUSY bit until the SDH register is set.
 1.89 12-Sep-1994  mycroft Turn off the delay debugging messages again.
 1.88 15-Aug-1994  mycroft Eliminate struct pte and struct pde.
 1.87 15-Aug-1994  mycroft Don't use altsts. Sigh.
 1.86 26-Jul-1994  mycroft Normalize label handling.
 1.85 21-Jul-1994  mycroft Return EINVAL if the requested transfer size is weird.
 1.84 16-Jun-1994  mycroft branches: 1.84.2;
b_un.b_addr -> b_data
 1.83 09-Jun-1994  pk Set `uio_procp' field in case we ever need it.
 1.82 05-May-1994  mycroft Remove now-bogus casts.
 1.81 05-May-1994  cgd lots of changes: prototype migration, move lots of variables, definitions,
and structure elements around. kill some unnecessary type and macro
definitions. standardize clock handling. More changes than you'd want.
 1.80 25-Apr-1994  mycroft Fix uninitialized pointer, pointed out by Michael VanLoon.
 1.79 24-Apr-1994  mycroft Don't need isa.h.
 1.78 20-Apr-1994  mycroft Use TAILQ_*() to maintain the drive activation queues.
 1.77 09-Apr-1994  mycroft Don't print a duplicate message about stray interrupts.
 1.76 07-Apr-1994  mycroft Implement dynamic IRQ configuration and IRQ sharing. Inline spl*() calls.
Reorganize and clean up the relevant code.
 1.75 30-Mar-1994  mycroft Turn off the timeout debugging info by default.
 1.74 29-Mar-1994  mycroft Updates for new autoconfig.
 1.73 12-Mar-1994  mycroft Prefix the busy-wait messages with `ignore this:' so people stop reporting
it as a bug.
 1.72 12-Mar-1994  mycroft Fix problem with reading the disk label if it's not at the beginning of the
disk.
 1.71 12-Mar-1994  mycroft Remove backward compatibility hacks for config(8), as they are no longer
useful.
 1.70 12-Mar-1994  mycroft Update to match config(8).
 1.69 11-Mar-1994  mycroft Various:
Put WDF_ERROR in the wdc_softc, to avoid gratuitously forcing non-active drives
into single-sector mode on the next transfer. Arrange to wait for an interrupt
after wdsetctlr(); this avoids long busy-wait delays, and gets rid of the
`extra interrupt' messages (except for one immediately after autoconfig on some
machines). Replace some uses of wdsetctlr() with simply lowering wd->sc_state;
no point in forcing this immediately. Allow control operations to time out.
Enable the warnings for long busy-wait delays by default. Some other minor
things not worth mentioning.
 1.68 10-Mar-1994  mycroft Add a heuristic for detecting old config(8)s. This is very temporary.
 1.67 10-Mar-1994  mycroft Now that the disk probe routines are actually called, they need to return
something.
 1.66 10-Mar-1994  mycroft More updates for new config.
 1.65 10-Mar-1994  mycroft Update for new config.
 1.64 10-Mar-1994  mycroft Cleanup to fit standard coding conventions, and *many* bugs fixed.
 1.63 07-Mar-1994  mycroft Some changes inspired by Bruce Evans's driver:
Consolidate error reporting in one function. Actually use the dk_status and
dk_error fields, and pass around only the error bit to avoid lots of unneeded
assignments and tests. No functional differences.
 1.62 07-Mar-1994  mycroft Remove DKFL_QUIET.
 1.61 06-Mar-1994  mycroft DELAY() --> delay(). This is not a macro.
 1.60 05-Mar-1994  mycroft Move the drive select and various other things into wdcommand(), to avoid
duplicating code.
 1.59 04-Mar-1994  mycroft Make sure we clear the interrupt when necessary. Why this isn't needed on
my hardware is a mystery.
 1.58 04-Mar-1994  mycroft Add another missing splx().
 1.57 04-Mar-1994  mycroft Remember to splx() on some error conditions.
 1.56 04-Mar-1994  mycroft Start the timeout loop *after* we've decided there is a disk there.
 1.55 04-Mar-1994  mycroft De-ANSIfy. Move the timeout counter into struct disk. Remove unneeded args
to wd[gs]etctlr(). Other minor changes.
 1.54 03-Mar-1994  mycroft Watch `seek complete' bit per IBM documentation. Clean up wdioctl(). Other
minor cleanup.
 1.53 02-Mar-1994  mycroft Be more careful about clearing the controller and drive active flags.
 1.52 02-Mar-1994  mycroft Don't busy-wait for read or write completion.
 1.51 02-Mar-1994  mycroft Make wdc_wait() do an implicit wait_for_unbusy(). Pass structure pointers
rahter than unit numbers where appropriate. Fix conflict if two drives have
I/O pending at the same time. Add some more sanity checks. Some other minor
cleanup.
 1.50 26-Feb-1994  mycroft Get the residual count right.
 1.49 26-Feb-1994  mycroft Clean up busy-wait code a little, and we need a special disksort() to skip
over bufs in a combined transfer.
 1.48 26-Feb-1994  mycroft Fix bug from 1.43 correctly.
 1.47 26-Feb-1994  mycroft Fix typo.
 1.46 26-Feb-1994  mycroft Various cleanup, and fix last night's bug.
 1.45 26-Feb-1994  mycroft Fix typo.
 1.44 26-Feb-1994  mycroft Some more stylistic cleanup.
 1.43 25-Feb-1994  mycroft Fix an obvious bug that could stall the queue.
 1.42 25-Feb-1994  mycroft Remove some redundant code.
 1.41 25-Feb-1994  mycroft Increase polling delay.
 1.40 25-Feb-1994  mycroft Generalize busy-wait loops.
 1.39 25-Feb-1994  mycroft Don't bother #ifdefing things that are needed and always enabled.
 1.38 25-Feb-1994  mycroft Don't reimplement disksort().
 1.37 25-Feb-1994  mycroft `We are the KNF. You will be assimilated.'
 1.36 10-Feb-1994  mycroft Back pointers are not useful for disksort().
 1.35 09-Feb-1994  mycroft All ioctl routines take a struct proc * now.
 1.34 06-Feb-1994  mycroft Remove another use of b_actl.
 1.33 06-Feb-1994  mycroft Use b_actf, not av_forw.
 1.32 03-Jan-1994  mycroft Inline all the [lr]cr[023]() `functions' in cpufunc.h.
 1.31 03-Jan-1994  mycroft Inline all the [lr]cr[023]() `functions' in cpufunc.h.
 1.30 24-Dec-1993  ws Correct dumping to WD (was destroyed by MACHINE_NONCONTIG code)
 1.29 20-Dec-1993  mycroft Canonicalize all #includes, and add pio.h where appropriate.
 1.28 16-Dec-1993  mycroft Eliminate reference to Maxmem.
 1.27 13-Dec-1993  cgd from Soeren Schmidt <sos@login.dkuug.dk>:
Basically it does a timeout on lost interrupts, starting the operation
again and logging and error message on the console.
additionally fixes some of the (newer ending) while loops
(that made it work with two IDE disks !)
 1.26 01-Aug-1993  mycroft branches: 1.26.2;
Add RCS identifiers (this time on the correct side of the branch), and
incorporate recent changes in netbsd-0-9 branch.
 1.25 28-Jul-1993  cgd incorporate changes from 0-9-base to 0-9-ALPHA
 1.24 08-Jul-1993  deraadt branches: 1.24.2;
documentation error
 1.23 07-Jul-1993  deraadt added a timeout debugging mode to wd.c so that people can see how slow
their controllers really are. Look for the WDCNDELAY_DEBUG definition.
 1.22 07-Jul-1993  deraadt ok, 5 second command timeouts..
 1.21 06-Jul-1993  deraadt number of changes:
1. controller reset code moved into one function, called from more places now.
2. more places now do timeouts. TIPCAT code turned on.
3. blew away the unused WDOPENLOCK test code.
4. delay is now done using WDCNDELAY repetitions of DELAY(25). Up to 2.5 second
pause accepted from controller (some controllers have some commands that are
actually this slow)
5. some of the old timeout code was really whacked.
 1.20 05-Jul-1993  deraadt added a missing {/} pair that barsoom seems to have taken out, but
which the standard 386bsd 0.1 driver did have. This is for a piece of
code in the bad block handling. found/fixed by hpeyerl.
 1.19 05-Jul-1993  deraadt all timeout code now calls DELAY() so the timeout delays are the same
length on all processors. Where possible, the controller is reset if it
times out.
 1.18 05-Jul-1993  deraadt increased the timeout inside wdcommand(). Some controllers impliment some
commands very slowly.
 1.17 18-Jun-1993  cgd fix the divide-by-zero problem w/old disks. this patch is ws's "fault."
 1.16 28-May-1993  deraadt 1. It is now possible to build a kernel that does not have a pc0 device driver.
2. "press any key to reboot" reads the key from the console.
3. wddump() still needs a non-blocking getc() routine (or flush)
 1.15 22-May-1993  cgd add rcsids to everything and clean up headers
 1.14 20-May-1993  deraadt First cut at cpu independent disklabels.
There will be niggly little details no doubt..
 1.13 10-May-1993  deraadt This patch allows the wd device driver to tell how big the disks are on
WD1007-derived controllers. In this example, wdc0 is a WD1007-clone,
and wdc1 is a WD1003-clone. WD1007 controllers are generally ESDI
and IDE controllers.
wdc0 at 0x1f0-0x1f7 irq 14 on isa
wd0 at wdc0 targ 0: 322MB 1224 cyl, 15 head, 36 sec <disktype>
wdc1 at 0x170-0x17f irq 7 on isa
wd2 at wdc1 targ 0: (unknown size) <disktype>
 1.12 09-May-1993  deraadt i thought i deleted this line before!
 1.11 04-May-1993  deraadt support for making dev->id_alive be set, this is for iostat to
find disk devices. wee bit of a kludge. sub-device attach()
routines must now return 1 for successful attach(), 0 otherwise.
Other bsd's do this too..
 1.10 18-Apr-1993  deraadt slight code quality improvement. not related to the bug with very
old controllers or disks! :-)
 1.9 15-Apr-1993  deraadt Once more: some changes to the ioconf.c structure. It will now handle
devices hanging off controllers, any kind of controller.
A device on isa0 is called at probe(self), then attach(self)
A controller on isa0 is called at probe(self), then all it's children with
defined unit numbers are initialized by calling attach(subdev); next all
subdevices with unit ? are initialized by calling attach(sundev).
Almost all device entry points is now like the vax/sun model (intr being
the weird one)
 1.8 15-Apr-1993  deraadt wd driver would fail on calling wdgetctlr() for older drives and
controllers. New behaviour is that if the standard method fails, try
to recalibrate the drive. Success means the drive exists. patch by
hpeyerl@novatel.cuc.ab.ca (who owns several of these historical
artifacts)
 1.7 12-Apr-1993  deraadt new scsi subsystem.
changes also in config/mkioconf.c & sys/scsi/*
 1.6 09-Apr-1993  cgd have probe return size of io space on successful return, rather
than simply 1.
 1.5 08-Apr-1993  deraadt dmesg output at boottime now tries to print out information as
soon as it is available. The output looks much more like Sunos.
 1.4 06-Apr-1993  deraadt ripped out a message
 1.3 06-Apr-1993  deraadt Mod-barsoom wd additions. config program + kernel config files changed too.
fd support had to change slightly too.
 1.2 21-Mar-1993  cgd after 0.2.2 "stable" patches applied
 1.1 21-Mar-1993  cgd branches: 1.1.1;
Initial revision
 1.1.1.1 21-Mar-1993  cgd initial import of 386bsd-0.1 sources
 1.24.2.3 28-Jul-1993  deraadt wd.c: fixes 1 drive systems
fd.c: improves reliability
changes from wolfgang
 1.24.2.2 22-Jul-1993  cgd fix comment for previous commit...
 1.24.2.1 20-Jul-1993  cgd change WDCNDELAY to 400000 -- delay -> 10s, at theo's request
 1.26.2.5 06-Feb-1994  mycroft Use b_actf, not av_forw.
 1.26.2.4 13-Dec-1993  cgd partial (hopefully correct) update from trunk
 1.26.2.3 27-Oct-1993  mycroft Minor cleanup.
 1.26.2.2 24-Sep-1993  mycroft clock.c: Finish conversion. Make rtcput() work if VRT bit not set. Speed up
time conversion in delay().
fd.c: Minor changes to autoconfig. Use dv_parent rather than storing another
pointer to the fdc_softc in each fd_softc. Change disk label stuff for our
kernel.
icu.s: Make splx() and splnone() return previous cpl for now.
isa.c: Minor autoconfig changes.
isa.h: rtcin() is now nvram() and is declare in nvram.h. Make isaphysmem a
caddr_t.
pccons.c: XXXX: Maybe we should insist they open /dev/io now.
timerreg.h: #define TIMER_NPORTS.
wd.c: Delete unused wdnoreloc() and wddospart(), and use remaining two bits in
unit number. struct disk --> struct wd_softc. struct board --> struct
wdc_softc. Still needs much work.
 1.26.2.1 14-Sep-1993  mycroft New i386 code.
 1.84.2.7 11-Oct-1994  mycroft Update from trunk.
 1.84.2.6 07-Oct-1994  mycroft Update from trunk.
 1.84.2.5 12-Sep-1994  cgd from trunk.
 1.84.2.4 15-Aug-1994  mycroft update from trunk
 1.84.2.3 15-Aug-1994  mycroft update from trunk
 1.84.2.2 26-Jul-1994  cgd update from trunk.
 1.84.2.1 22-Jul-1994  cgd from trunk.
 1.156.6.1 13-May-1997  thorpej Adapt to new software interface to the on-board ISA DMA controller.
 1.160.2.3 17-Jul-1997  bouyer Use "locator.h".
 1.160.2.2 17-Jul-1997  bouyer Update from trunck.
 1.160.2.1 01-Jul-1997  bouyer Add the ide atapi driver from my original atapi driver, ported
to scsipi.
 1.162.2.2 14-Oct-1997  thorpej Update marc-pcmcia branch from trunk.
 1.162.2.1 27-Aug-1997  thorpej Update marc-pcmcia branch from trunk.
 1.175.2.13 04-Oct-1998  bouyer atavar.h: drv_softc is a struct device * instead of void*, as it's mostly
used for dv_xname
wd.c: convert for drv_softc type change, printf cleanup
wdc.c: always call ata_get_params() (params was used initialised with
non-32bit controllers, leaving to bogus PIO/DMA mode report).
Cleaup of the PIO/DMA mode message.
pciide_piix_reg.h: Fix definition of PIIX_IDETIM_CLEAR (unused before)
pciide.c: add a method do disable a channel on know device. If a channel
doesn't have any drive, we disable it and free its resources if disable
was successfull. This should help with laptops where the second channel
of the PIIX4 is unused but not disabled by BIOS. On such laptops,
irq15 can be used for PCMCIA but it was claimed by pciide.
Misc printf cleanup.
wdc_isa.c: printf cleanup.
 1.175.2.12 20-Sep-1998  bouyer Kill the 'old drive' flag. The probe for IDE controllers now looks
like this :
- if status = 0xff for both master and slave, no drive at all -> exit.
- issue a reset (timeout = 31s).
- test signatures: if ATAPI signature, flag a (possible) ATAPI devices here
else test register writability, then wait for the drive to become
ready, issue a diagnostic command and wait for the drive to become ready
if ok, flag a real ATA device.
This looks more like the old probe.
For ATA devices, if IDENTIFY fails, then it's an old drive and use
a fake disklabel/geometry.
 1.175.2.11 20-Sep-1998  bouyer Init up xfer->drive in wdc_ata_bio(), or bad things will happen for drive 1.
No need to init xfer->channel, it's done in wdc_exec_xfer().
 1.175.2.10 20-Sep-1998  bouyer - always call config_found() from wdcattach(), instead of printing our own
the "not configured" message ourselve. When no atapibus is configured,
use a fake ata_atapi_attach, else call wdc_atapibus_attach(). This way we
don't have to include the whole atapi support in an ata-only config.
- the dma_* function pointers take a 'int flags' as last argument, instead
of an 'int read'. The 2 valid flags are: WDC_DMA_READ (to select read/write)
and WDC_DMA_POLL, to signal interrupt-less mode (for core dumps).
- Reworked wdcprobe() so look more like the old one. A status of 0xff is
interpreted as "no drive" (freebsd does this); this this speed up
the probe for non-IDE machines (a reset timeout has been bumped to 31s,
to match the specs). The probe set ups the drive flags to either ATA or
ATAPI (depend on register signature) or OLD (if register signature don't
match, but a working drive appears to be there). Later the ATA code
will look at the OLD flag to decide wether there is no drive, or an old
(ST506) one if WDC_IDENTIFY fails.
- For known PCI chips, don't use the wdcprobe() heuristic to find wether
a channel is enabled/disabled; use the chip's specific registers for this.
- Clear the command queue when we have a polled command. This allows dumps to
work even if a command was being handled by the controller at panic time.
- Increase the number of available DMA segments by one, as user requests may
not be aligned on a page boundary.
 1.175.2.9 11-Sep-1998  bouyer Sync with HEAD.
 1.175.2.8 21-Aug-1998  bouyer Sync with -current
 1.175.2.7 13-Aug-1998  bouyer - sync with HEAD
- better error detection and report for wd drives
- better wddump() support
- wdcwait() now takes a timout argument, so that we can honnor the
timeout argument of scsipi cmds.
 1.175.2.6 13-Jun-1998  bouyer Sync with HEAD.
 1.175.2.5 10-Jun-1998  bouyer Print the minimum and recomended DMA timings if DEBUG_PROBE is defined.
 1.175.2.4 09-Jun-1998  bouyer - sync with trunck
- don't try to set the drive's PIO mode if we don't know how to set it on the
controller. This seems to confuse some 'intelligent' ISA controllers.
 1.175.2.3 05-Jun-1998  bouyer - call wdc_ata_bio_done() only once.
- better error recovery: if an error occured, retry the transfer in
single sector PIO mode.
 1.175.2.2 04-Jun-1998  bouyer Commit changes to the IDE system in a branch. This allows a better separation
between higth-level and low-level (i.e. registers read/write) and generalize
the queue for all commands. This also add supports for IDE DMA.
 1.175.2.1 04-Jun-1998  bouyer file wd.c was added on branch bouyer-ide on 1998-06-04 16:52:35 +0000
 1.193.2.1 07-Jul-2000  he Apply patch (requested by bouyer):
Add support for the following PCIIDE controllers:
o AMD 756
o CMD PCI0648 and PCI0649
o Hightpoint HPT366
o OPTi 82c621 (and a few of its derivatives)
o Promise Ultra/33 and Ultra/66
o Intel 82801 (ICH/ICH0)
Also fix PR#10437 (detect more ATAPI devices).
 1.196.6.2 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.196.6.1 21-Dec-1999  wrstuden Initial commit of recent changes to make DEV_BSIZE go away.

Runs on i386, needs work on other arch's. Main kernel routines should be
fine, but a number of the stand programs need help.

cd, fd, ccd, wd, and sd have been updated. sd has been tested with non-512
byte block devices. vnd, raidframe, and lfs need work.

Non 2**n block support is automatic for LKM's and conditional for kernels
on "options NON_PO2_BLOCKS".
 1.196.4.1 15-Nov-1999  fvdl Sync with -current
 1.196.2.4 18-Jan-2001  bouyer Sync with head (for UBC+NFS fixes, mostly).
 1.196.2.3 05-Jan-2001  bouyer Sync with HEAD
 1.196.2.2 20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
A i386 GENERIC kernel compiles without the siop, ahc and bha drivers
(will be updated later). i386 IDE/ATAPI and ncr work, as well as
sparc/esp_sbus. alpha should work as well (untested yet).
siop, ahc and bha will be updated once I've updated the branch to current
-current, as well as machine-dependant code.
 1.196.2.1 20-Oct-1999  thorpej Sync w/ trunk.
 1.204.2.1 22-Jun-2000  minoura Sync w/ netbsd-1-5-base.
 1.205.2.2 06-May-2001  he Pull up revision 1.213 (requested by fvdl):
Compensate for the fact that after __HAVE_OLD_DISKLABEL,
DISKMINOR(u,p) is no longer contiguous over ``p'', so ranges
cannot be passed to vdevgone().
 1.205.2.1 01-May-2001  he Pull up revisions 1.211-1.212 (requested by fvdl):
Increase the number of BSD disklabel partitions on i386 to 16.
 1.212.2.10 07-Jan-2003  thorpej Sync with HEAD.
 1.212.2.9 19-Dec-2002  thorpej Sync with HEAD.
 1.212.2.8 11-Nov-2002  nathanw Catch up to -current
 1.212.2.7 18-Oct-2002  nathanw Catch up to -current.
 1.212.2.6 17-Sep-2002  nathanw Catch up to -current.
 1.212.2.5 01-Aug-2002  nathanw Catch up to -current.
 1.212.2.4 28-Feb-2002  nathanw Catch up to -current.
 1.212.2.3 08-Jan-2002  nathanw Catch up to -current.
 1.212.2.2 14-Nov-2001  nathanw Catch up to -current.
 1.212.2.1 21-Jun-2001  nathanw Catch up to -current.
 1.214.4.3 26-Sep-2001  fvdl * add a VCLONED vnode flag that indicates a vnode representing a cloned
device.
* rename REVOKEALL to REVOKEALIAS, and add a REVOKECLONE flag, to pass
to VOP_REVOKE
* the revoke system call will revoke all aliases, as before, but not the
clones
* vdevgone is called when detaching a device, so make it use REVOKECLONE
to get rid of all clones as well
* clean up all uses of VOP_OPEN wrt. locking.
* add a few VOPS to spec_vnops that need to do something when it's a
clone vnode (access and getattr)
* add a copy of the vnode vattr structure of the original 'master' vnode
to the specinfo of a cloned vnode. could possibly redirect getattr to
the 'master' vnode, but this has issues with revoke
* add a vdev_reassignvp function that disassociates a vnode from its
original device, and reassociates it with the specified dev_t. to be
used by cloning devices only, in case a new minor is allocated.
* change all direct references in drivers to v_devcookie and v_rdev
to vdev_privdata(vp) and vdev_rdev(vp). for diagnostic purposes
when debugging race conditions that still exist wrt. locking and
revoking vnodes.
* make the locking state of a vnode consistent when passed to
d_open and d_close (unlocked). locked would be better, but has
some deadlock issues
 1.214.4.2 18-Sep-2001  fvdl Various changes to make cloning devices possible:

* Add an extra argument (struct vnode **) to VOP_OPEN. If it is
not NULL, specfs will create a cloned (aliased) vnode during
the call, and return it there. The caller should release and
unlock the original vnode if a new vnode was returned. The
new vnode is returned locked.

* Add a flag field to the cdevsw and bdevsw structures.
DF_CLONING indicates that it wants a new vnode for each
open (XXX is there a better way? devprop?)

* If a device is cloning, always call the close entry
point for a VOP_CLOSE.


Also, rewrite cons.c to do the right thing with vnodes. Use VOPs
rather then direct device entry calls. Suggested by mycroft@

Light to moderate testing done an i386 system (arch doesn't matter
though, these are MI changes).
 1.214.4.1 07-Sep-2001  thorpej Commit my "devvp" changes to the thorpej-devvp branch. This
replaces the use of dev_t in most places with a struct vnode *.

This will form the basic infrastructure for real cloning device
support (besides being architecurally cleaner -- it'll be good
to get away from using numbers to represent objects).
 1.214.2.4 10-Oct-2002  jdolecek sync kqueue with -current; this includes merge of gehenna-devsw branch,
merge of i386 MP branch, and part of autoconf rototil work
 1.214.2.3 06-Sep-2002  jdolecek sync kqueue branch with HEAD
 1.214.2.2 11-Feb-2002  jdolecek Sync w/ -current.
 1.214.2.1 10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.220.10.7 14-May-2004  jdc Pull up revision 1.275 via patch (requested by bouyer in ticket #1691)

When using LBA48, use Flush cache extended command if supported.
From Christos Zoulas in PR 25366, with minor adjustements from me.
 1.220.10.6 28-Mar-2004  jmc Pullup rev 1.260 via patch (requested by bouyer in ticket #1629)

For LBA drives, compute the number of cylinders from the drive capacity
in the default disklabel and the boot message, instead of using the
value reported by the drive (which is 16383 if the drive is larger than 8G).
Fixes PR#9864
 1.220.10.5 28-Mar-2004  jmc Pullup rev 1.258 via patch (requested by bouyer in ticket #1628)

Add LBA48_THRESHOLD #define and set it to the *correct* value. Use it
in place of 0xffffff (which is one too few 'f').
 1.220.10.4 15-Aug-2003  tron Pull up revision 1.256 (requested by bouyer in ticket #1370):
Don't use LBA48 if we don't have to (that is, if the request is inside the
first 128GB). Tested by Andreas Johansson.
 1.220.10.3 30-Jun-2003  grant Apply patch (requested by bouyer in ticket #1354):

Add support for the getcache/setcache ioctls.

Acceptable cache operations are r and rw, not none and w
Properly initialise the timeout value.

Fix typo.
 1.220.10.2 22-Jul-2002  lukem Pull up revision 1.222 (requested by yamt in ticket #536):
do previous differently for wddump case.
 1.220.10.1 22-Jul-2002  lukem Pull up revision 1.221 (requested by yamt in ticket #536):
- remove big buffers on stack. discussed on tech-kern.
- constify.
 1.220.8.4 29-Aug-2002  gehenna catch up with -current.
 1.220.8.3 20-Jul-2002  gehenna catch up with -current.
 1.220.8.2 15-Jul-2002  gehenna catch up with -current.
 1.220.8.1 16-May-2002  gehenna Add the block/character device switches.
Replace the access to devsw table with calling devsw API.
 1.257.2.14 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.257.2.13 01-Apr-2005  skrll Sync with HEAD.
 1.257.2.12 09-Mar-2005  skrll Adapt commented out code to branch.
 1.257.2.11 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.257.2.10 29-Nov-2004  skrll Sync with HEAD.
 1.257.2.9 02-Nov-2004  skrll Sync with HEAD.
 1.257.2.8 19-Oct-2004  skrll Sync with HEAD
 1.257.2.7 21-Sep-2004  skrll Fix the sync with head I botched.
 1.257.2.6 18-Sep-2004  skrll Sync with HEAD.
 1.257.2.5 03-Sep-2004  skrll Sync with HEAD
 1.257.2.4 25-Aug-2004  skrll Sync with HEAD.
 1.257.2.3 12-Aug-2004  skrll Sync with HEAD.
 1.257.2.2 03-Aug-2004  skrll Sync with HEAD
 1.257.2.1 02-Jul-2003  darrenr Apply the aborted ktrace-lwp changes to a specific branch. This is just for
others to review, I'm concerned that patch fuziness may have resulted in some
errant code being generated but I'll look at that later by comparing the diff
from the base to the branch with the file I attempt to apply to it. This will,
at the very least, put the changes in a better context for others to review
them and attempt to tinker with removing passing of 'struct lwp' through
the kernel.
 1.274.2.14 15-Oct-2007  riz Pull up following revision(s) (requested by bouyer in ticket #11369):
sys/dev/ata/wd.c: revision 1.345
Add a workaround for drives with the LBA48 bug:
if we get a "ID not found" error for a transfer crossing LBA48_THRESHOLD,
and the drive is larger than 128GB, automatically add WD_QUIRK_FORCE_LBA48
to the drive's quirks and retry the transfers.
Hopefully this will obsolete the WD_QUIRK_FORCE_LBA48 quirk list ...
 1.274.2.13 20-Nov-2006  tron Pull up following revision(s) (requested by bouyer in ticket #10747):
sys/dev/ata/wd.c: revision 1.334
Yet another broken seagate drive.
 1.274.2.12 20-Nov-2006  tron Pull up following revision(s) (requested by bouyer in ticket #10746):
sys/dev/ata/wd.c: revision 1.326
Add yet another broken 160GB seagate drive. Mine. ST3160021A.
 1.274.2.11 06-Aug-2006  tron Pull up following revision(s) (requested by bouyer in ticket #10644):
sys/dev/ata/wd.c: revision 1.325 via patch
Adjust quirk entry for seagate drives. Should fix kern/33637 by Shigeya
Suzuki.
 1.274.2.10 24-May-2006  riz Pull up following revision(s) (requested by bouyer in ticket #10625):
sys/dev/ata/wd.c: revision 1.324 via patch
Add yet another broken 160Gb seagate drive. I though we had found all of
them ... reported by Damian Lubosch on current-users.
 1.274.2.9 28-Mar-2006  riz Pull up patch (requested by bouyer in ticket #10397):
sync the list of quirky seagate drives with -current
 1.274.2.8 01-Oct-2004  jmc branches: 1.274.2.8.2;
Pullup rev 1.295 (requested by bouyer in ticket #882)

Add a WD_QUIRK_FORCE_LBA48 quirck, which forces all access to use LBA48.
Use it for ST3200822A, which has a broken firmware and return an error
when accessing sector 0xfffffff with a LBA command.
 1.274.2.7 17-Sep-2004  jmc Pullup patch (requested by bouyer in ticket #853)

In case of error, reset the channel with AT_RST_NOCMD. We know the reset
wont abort any command at this point.
 1.274.2.6 11-Aug-2004  jmc Pullup rev 1.282-1.283 (requested by bouyer in ticket #735)

Move cleaning of the bad block list in wddetach() after the call to
disk_detach(), to avoid a race condition with the interrupt code.
wddetach(): move lockmgr(LK_DRAIN) near the end; otherwise we'll panic in
wddone() (vdevgone() may end up calling it) trying to get the lock.
wdopen(): check for DVF_ACTIVE.
 1.274.2.5 11-Aug-2004  jmc Pullup rev 1.281 (requested by bouyer in ticket #733)

Implement asynchronous channel reset.
Use this to reset the channel before doing a dump, instead of the hack in
wdc_exec_xfer() based on C_POLL. This hack was causing problems on
controllers with a shared queue, because we now can have C_POLL set during
concurent channels probes
 1.274.2.4 23-Jul-2004  tron Pull up revision 1.278 (requested by mycroft in ticket #704):
Fix an extremely obvious bug in the handling of the bad block list: the "max"
block was being set 512x further out than it should be, causing rather severe
escalation of the error.
 1.274.2.3 02-Jul-2004  he Pull up revisions 1.279-1.280 (requested by mycroft in ticket #572):
Several fixes mostly related to USB:
o Don't force ATA_SINGLE on devices which don't support
READ/WRITE MULTI. Increases performance with several
CF cards substantially.
o Update copyright.
 1.274.2.2 29-May-2004  tron Pull up revision 1.276 (requested by bouyer in ticket #398):
Add support for the DIOCCACHESYNC ioctl.
 1.274.2.1 09-May-2004  jdc Pull up revision 1.275 (requested by bouyer in ticket #262)

When using LBA48, use Flush cache extended command if supported.
From Christos Zoulas in PR 25366, with minor adjustements from me.
 1.274.2.8.2.12 15-Oct-2007  riz Pull up following revision(s) (requested by bouyer in ticket #11369):
sys/dev/ata/wd.c: revision 1.345
Add a workaround for drives with the LBA48 bug:
if we get a "ID not found" error for a transfer crossing LBA48_THRESHOLD,
and the drive is larger than 128GB, automatically add WD_QUIRK_FORCE_LBA48
to the drive's quirks and retry the transfers.
Hopefully this will obsolete the WD_QUIRK_FORCE_LBA48 quirk list ...
 1.274.2.8.2.11 20-Nov-2006  tron Pull up following revision(s) (requested by bouyer in ticket #10747):
sys/dev/ata/wd.c: revision 1.334
Yet another broken seagate drive.
 1.274.2.8.2.10 20-Nov-2006  tron Pull up following revision(s) (requested by bouyer in ticket #10746):
sys/dev/ata/wd.c: revision 1.326
Add yet another broken 160GB seagate drive. Mine. ST3160021A.
 1.274.2.8.2.9 06-Aug-2006  tron Pull up following revision(s) (requested by bouyer in ticket #10644):
sys/dev/ata/wd.c: revision 1.325 via patch
Adjust quirk entry for seagate drives. Should fix kern/33637 by Shigeya
Suzuki.
 1.274.2.8.2.8 24-May-2006  riz Pull up following revision(s) (requested by bouyer in ticket #10625):
sys/dev/ata/wd.c: revision 1.324
Add yet another broken 160Gb seagate drive. I though we had found all of
them ... reported by Damian Lubosch on current-users.
 1.274.2.8.2.7 28-Mar-2006  riz Pull up following revision(s) (requested by bouyer in ticket #10396):
sys/dev/ata/wd.c: revision 1.310
Use a pmatch(9) expression which should catch all present and future
seagate drives larger than 200GB for the WD_QUIRK_FORCE_LBA48 quirks.
Based on infos from
http://www.seagate.com/support/kb/disc/howto/interpret_model.html
 1.274.2.8.2.6 06-Sep-2005  riz branches: 1.274.2.8.2.6.2;
Pull up following revision(s) (requested by bouyer in ticket #5822):
sys/dev/ata/wd.c: revision 1.309
Add quirk for ST3200826A. From rudolf (netbsd at eq.cz) on tech-kern.
 1.274.2.8.2.5 06-Sep-2005  riz Pull up following revision(s) (requested by bouyer in ticket #5821):
sys/dev/ata/wd.c: revision 1.307
ST3300831A also needs WD_QUIRK_FORCE_LBA48.
 1.274.2.8.2.4 05-Jul-2005  riz Pull up revision 1.304 (requested by bouyer in ticket #2028):
The ST3160023A* also need the LBA48 quirk. From Timo Schoeler.
 1.274.2.8.2.3 05-Jul-2005  riz Pull up revision 1.303 via patch (requested by bouyer in ticket #2027):
Use pmatch() metacharacters to match both SATA and PATA versions of seagate
drives in the quirk table.
 1.274.2.8.2.2 16-May-2005  riz Pull up revision 1.300 (requested by bouyer in ticket #1522):
Add another broken seagate device to the quirk list.
Problem reported by rudolf <netbsd at eq.cz> on netbsd-help@
 1.274.2.8.2.1 06-Feb-2005  jmc Pullup rev 1.297 (requested by bouyer in ticket #986)

Some drives reports ATA version < 4, and still support (and need) the
WDCC_FLUSHCACHE command. So for drive < ATA4, test WDC_CMD2_FC in atap_cmd_set2
before returning ENODEV.
 1.274.2.8.2.6.2.6 15-Oct-2007  riz Pull up following revision(s) (requested by bouyer in ticket #11369):
sys/dev/ata/wd.c: revision 1.345
Add a workaround for drives with the LBA48 bug:
if we get a "ID not found" error for a transfer crossing LBA48_THRESHOLD,
and the drive is larger than 128GB, automatically add WD_QUIRK_FORCE_LBA48
to the drive's quirks and retry the transfers.
Hopefully this will obsolete the WD_QUIRK_FORCE_LBA48 quirk list ...
 1.274.2.8.2.6.2.5 20-Nov-2006  tron Pull up following revision(s) (requested by bouyer in ticket #10747):
sys/dev/ata/wd.c: revision 1.334
Yet another broken seagate drive.
 1.274.2.8.2.6.2.4 20-Nov-2006  tron Pull up following revision(s) (requested by bouyer in ticket #10746):
sys/dev/ata/wd.c: revision 1.326
Add yet another broken 160GB seagate drive. Mine. ST3160021A.
 1.274.2.8.2.6.2.3 06-Aug-2006  tron Pull up following revision(s) (requested by bouyer in ticket #10644):
sys/dev/ata/wd.c: revision 1.325 via patch
Adjust quirk entry for seagate drives. Should fix kern/33637 by Shigeya
Suzuki.
 1.274.2.8.2.6.2.2 24-May-2006  riz Pull up following revision(s) (requested by bouyer in ticket #10625):
sys/dev/ata/wd.c: revision 1.324
Add yet another broken 160Gb seagate drive. I though we had found all of
them ... reported by Damian Lubosch on current-users.
 1.274.2.8.2.6.2.1 28-Mar-2006  riz Pull up following revision(s) (requested by bouyer in ticket #10396):
sys/dev/ata/wd.c: revision 1.310
Use a pmatch(9) expression which should catch all present and future
seagate drives larger than 200GB for the WD_QUIRK_FORCE_LBA48 quirks.
Based on infos from
http://www.seagate.com/support/kb/disc/howto/interpret_model.html
 1.297.6.1 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.297.4.1 29-Apr-2005  kent sync with -current
 1.298.2.15 08-Apr-2008  jdc Pull up revision 1.359 (requested by bouyer in ticket #1910).

The Hitachi HDP725025GLA380 returns "aborted command" instead of
"id not found" when hitting the LBA48 bug, so also install the
LBA48 bug workaround on "aborted command" errors.
 1.298.2.14 15-Oct-2007  riz Pull up following revision(s) (requested by bouyer in ticket #1848):
sys/dev/ata/wd.c: revision 1.345
Add a workaround for drives with the LBA48 bug:
if we get a "ID not found" error for a transfer crossing LBA48_THRESHOLD,
and the drive is larger than 128GB, automatically add WD_QUIRK_FORCE_LBA48
to the drive's quirks and retry the transfers.
Hopefully this will obsolete the WD_QUIRK_FORCE_LBA48 quirk list ...
 1.298.2.13 20-Nov-2006  ghen Pull up following revision(s) (requested by bouyer in ticket #1583):
sys/dev/ata/wd.c: revision 1.334
Yet another broken seagate drive.
 1.298.2.12 30-Jun-2006  ghen branches: 1.298.2.12.2;
Pull up following revision(s) (requested by gendalia in ticket #1396):
sys/dev/ata/wd.c: revision 1.326
Add yet another broken 160GB seagate drive. Mine. ST3160021A.
 1.298.2.11 12-Jun-2006  tron Pull up following revision(s) (requested by bouyer in ticket #1375):
sys/dev/ata/wd.c: revision 1.325 via patch
Adjust quirk entry for seagate drives. Should fix kern/33637 by Shigeya
Suzuki.
 1.298.2.10 24-May-2006  riz Pull up following revision(s) (requested by bouyer in ticket #1337):
sys/dev/ata/wd.c: revision 1.324
Add yet another broken 160Gb seagate drive. I though we had found all of
them ... reported by Damian Lubosch on current-users.
 1.298.2.9 28-Mar-2006  riz Pull up following revision(s) (requested by bouyer in ticket #1227):
sys/dev/ata/wd.c: revision 1.310
Use a pmatch(9) expression which should catch all present and future
seagate drives larger than 200GB for the WD_QUIRK_FORCE_LBA48 quirks.
Based on infos from
http://www.seagate.com/support/kb/disc/howto/interpret_model.html
 1.298.2.8 27-Jan-2006  tron Pull up following revision(s) (requested by abs in ticket #1132):
sys/dev/ata/wd.c: revision 1.318
cast 16bit atap_capacity to u_int64_t before << 16. (assigned to
a u_int64_t variable). Fixes 1 to 2 TB disks in qemu. Tested with
1=0000GB and to a lesser extent 2047GB sparse files. For reference
an empty FFS1 filesystem on a 1200GB sparse file uses 19GB of disk.
 1.298.2.7 08-Sep-2005  tron branches: 1.298.2.7.2;
Pull up following revision(s) (requested by bouyer in ticket #750):
sys/dev/ata/wd.c: revision 1.309
Add quirk for ST3200826A. From rudolf (netbsd at eq.cz) on tech-kern.
 1.298.2.6 08-Sep-2005  tron Pull up following revision(s) (requested by bouyer in ticket #749):
sys/dev/ata/wd.c: revision 1.307
ST3300831A also needs WD_QUIRK_FORCE_LBA48.
 1.298.2.5 02-Jul-2005  tron Pull up revision 1.304 (requested by bouyer in ticket #497):
The ST3160023A* also need the LBA48 quirk. From Timo Schoeler.
 1.298.2.4 02-Jul-2005  tron Pull up revision 1.303 (requested by bouyer in ticket #496):
Use pmatch() metacharacters to match both SATA and PATA versions of seagate
drives in the quirk table.
 1.298.2.3 16-Jun-2005  tron Pull up revision 1.302 (requested by martti in ticket #459):
Added LBA48 quirk for Seagate ST3200822AS. Tested by Teemu Rinta-aho.
 1.298.2.2 19-May-2005  tron Pull up revision 1.300 (requested by bouyer in ticket #310):
Add another broken seagate device to the quirk list.
Problem reported by rudolf <netbsd at eq.cz> on netbsd-help@
 1.298.2.1 06-Apr-2005  tron Pull up revision 1.299 (requested by yamt in ticket #112):
introduce a function to drain bufq and use it where appropriate.
 1.298.2.12.2.3 08-Apr-2008  jdc Pull up revision 1.359 (requested by bouyer in ticket #1910).

The Hitachi HDP725025GLA380 returns "aborted command" instead of
"id not found" when hitting the LBA48 bug, so also install the
LBA48 bug workaround on "aborted command" errors.
 1.298.2.12.2.2 15-Oct-2007  riz Pull up following revision(s) (requested by bouyer in ticket #1848):
sys/dev/ata/wd.c: revision 1.345
Add a workaround for drives with the LBA48 bug:
if we get a "ID not found" error for a transfer crossing LBA48_THRESHOLD,
and the drive is larger than 128GB, automatically add WD_QUIRK_FORCE_LBA48
to the drive's quirks and retry the transfers.
Hopefully this will obsolete the WD_QUIRK_FORCE_LBA48 quirk list ...
 1.298.2.12.2.1 20-Nov-2006  ghen Pull up following revision(s) (requested by bouyer in ticket #1583):
sys/dev/ata/wd.c: revision 1.334
Yet another broken seagate drive.
 1.298.2.7.2.8 08-Apr-2008  jdc Pull up revision 1.359 (requested by bouyer in ticket #1910).

The Hitachi HDP725025GLA380 returns "aborted command" instead of
"id not found" when hitting the LBA48 bug, so also install the
LBA48 bug workaround on "aborted command" errors.
 1.298.2.7.2.7 15-Oct-2007  riz Pull up following revision(s) (requested by bouyer in ticket #1848):
sys/dev/ata/wd.c: revision 1.345
Add a workaround for drives with the LBA48 bug:
if we get a "ID not found" error for a transfer crossing LBA48_THRESHOLD,
and the drive is larger than 128GB, automatically add WD_QUIRK_FORCE_LBA48
to the drive's quirks and retry the transfers.
Hopefully this will obsolete the WD_QUIRK_FORCE_LBA48 quirk list ...
 1.298.2.7.2.6 20-Nov-2006  ghen Pull up following revision(s) (requested by bouyer in ticket #1583):
sys/dev/ata/wd.c: revision 1.334
Yet another broken seagate drive.
 1.298.2.7.2.5 20-Nov-2006  ghen Pull up following revision(s) (requested by bouyer in ticket #1582):
sys/dev/ata/wd.c: revision 1.326
Add yet another broken 160GB seagate drive. Mine. ST3160021A.
 1.298.2.7.2.4 12-Jun-2006  tron Pull up following revision(s) (requested by bouyer in ticket #1375):
sys/dev/ata/wd.c: revision 1.325 via patch
Adjust quirk entry for seagate drives. Should fix kern/33637 by Shigeya
Suzuki.
 1.298.2.7.2.3 24-May-2006  riz Pull up following revision(s) (requested by bouyer in ticket #1337):
sys/dev/ata/wd.c: revision 1.324
Add yet another broken 160Gb seagate drive. I though we had found all of
them ... reported by Damian Lubosch on current-users.
 1.298.2.7.2.2 28-Mar-2006  riz Pull up following revision(s) (requested by bouyer in ticket #1227):
sys/dev/ata/wd.c: revision 1.310
Use a pmatch(9) expression which should catch all present and future
seagate drives larger than 200GB for the WD_QUIRK_FORCE_LBA48 quirks.
Based on infos from
http://www.seagate.com/support/kb/disc/howto/interpret_model.html
 1.298.2.7.2.1 27-Jan-2006  tron Pull up following revision(s) (requested by abs in ticket #1132):
sys/dev/ata/wd.c: revision 1.318
cast 16bit atap_capacity to u_int64_t before << 16. (assigned to
a u_int64_t variable). Fixes 1 to 2 TB disks in qemu. Tested with
1=0000GB and to a lesser extent 2047GB sparse files. For reference
an empty FFS1 filesystem on a 1200GB sparse file uses 19GB of disk.
 1.305.2.10 24-Mar-2008  yamt sync with head.
 1.305.2.9 17-Mar-2008  yamt sync with head.
 1.305.2.8 27-Feb-2008  yamt sync with head.
 1.305.2.7 21-Jan-2008  yamt sync with head
 1.305.2.6 15-Nov-2007  yamt sync with head.
 1.305.2.5 27-Oct-2007  yamt sync with head.
 1.305.2.4 03-Sep-2007  yamt sync with head.
 1.305.2.3 26-Feb-2007  yamt sync with head.
 1.305.2.2 30-Dec-2006  yamt sync with head.
 1.305.2.1 21-Jun-2006  yamt sync with head.
 1.312.2.1 02-Nov-2005  yamt sync with head.
 1.315.2.4 01-Mar-2006  yamt sync with head.
 1.315.2.3 05-Feb-2006  yamt adapt wd.
 1.315.2.2 01-Feb-2006  yamt sync with head.
 1.315.2.1 15-Jan-2006  yamt sync with head.
 1.318.4.3 07-Jun-2006  kardel Sync with head.
 1.318.4.2 01-Jun-2006  kardel Sync with head.
 1.318.4.1 22-Apr-2006  simonb Sync with head.
 1.318.2.1 09-Sep-2006  rpaulo sync with head
 1.320.6.2 24-May-2006  tron Merge 2006-05-24 NetBSD-current into the "peter-altq" branch.
 1.320.6.1 31-Mar-2006  tron Merge 2006-03-31 NetBSD-current into the "peter-altq" branch.
 1.320.4.1 19-Apr-2006  elad sync with head.
 1.320.2.4 11-Aug-2006  yamt sync with head
 1.320.2.3 26-Jun-2006  yamt sync with head.
 1.320.2.2 24-May-2006  yamt sync with head.
 1.320.2.1 01-Apr-2006  yamt sync with head.
 1.324.2.1 19-Jun-2006  chap Sync with head.
 1.325.2.1 13-Jul-2006  gdamore Merge from HEAD.
 1.329.4.2 10-Dec-2006  yamt sync with head.
 1.329.4.1 22-Oct-2006  yamt sync with head
 1.329.2.2 06-Feb-2007  ad Quieten noisy boot messages.
 1.329.2.1 18-Nov-2006  ad Sync with head.
 1.335.4.2 03-Jun-2008  skrll Sync with netbsd-4.
 1.335.4.1 30-Sep-2007  wrstuden Catch up on netbsd-4 as of a few days ago.
 1.335.2.2 30-Mar-2008  jdc Pull up revision 1.359 (requested by bouyer in ticket #1094).

The Hitachi HDP725025GLA380 returns "aborted command" instead of
"id not found" when hitting the LBA48 bug, so also install the
LBA48 bug workaround on "aborted command" errors.
 1.335.2.1 25-Sep-2007  xtraeme branches: 1.335.2.1.4;
Pull up following revision(s) (requested by bouyer in ticket #899):
sys/dev/ata/wd.c: revision 1.345

Add a workaround for drives with the LBA48 bug:
if we get a "ID not found" error for a transfer crossing LBA48_THRESHOLD,
and the drive is larger than 128GB, automatically add WD_QUIRK_FORCE_LBA48
to the drive's quirks and retry the transfers.
Hopefully this will obsolete the WD_QUIRK_FORCE_LBA48 quirk list ...
 1.335.2.1.4.1 30-Mar-2008  jdc Pull up revision 1.359 (requested by bouyer in ticket #1094).

The Hitachi HDP725025GLA380 returns "aborted command" instead of
"id not found" when hitting the LBA48 bug, so also install the
LBA48 bug workaround on "aborted command" errors.
 1.336.2.2 15-Apr-2007  yamt sync with head.
 1.336.2.1 12-Mar-2007  rmind Sync with HEAD.
 1.337.4.1 11-Jul-2007  mjf Sync with head.
 1.337.2.11 23-Oct-2007  ad Sync with head.
 1.337.2.10 09-Oct-2007  ad Sync with head.
 1.337.2.9 24-Aug-2007  ad Sync with buffer cache locking changes. See buf.h/vfs_bio.c for details.
Some minor portions are incomplete and needs to be verified as a whole.
 1.337.2.8 20-Aug-2007  ad - Alter disk attach/detach to fix a panic when closing a vnd device.
- Sync with HEAD.
 1.337.2.7 19-Aug-2007  ad - Back out the biodone() changes.
- Eliminate B_ERROR (from HEAD).
 1.337.2.6 15-Jul-2007  ad Sync with head.
 1.337.2.5 01-Jul-2007  ad Adapt to callout API change.
 1.337.2.4 17-Jun-2007  ad - Increase the number of thread priorities from 128 to 256. How the space
is set up is to be revisited.
- Implement soft interrupts as kernel threads. A generic implementation
is provided, with hooks for fast-path MD code that can run the interrupt
threads over the top of other threads executing in the kernel.
- Split vnode::v_flag into three fields, depending on how the flag is
locked (by the interlock, by the vnode lock, by the file system).
- Miscellaneous locking fixes and improvements.
 1.337.2.3 13-May-2007  ad - Pass the error number and residual count to biodone(), and let it handle
setting error indicators. Prepare to eliminate B_ERROR.
- Add a flag argument to brelse() to be set into the buf's flags, instead
of doing it directly. Typically used to set B_INVAL.
- Add a "struct cpu_info *" argument to kthread_create(), to be used to
create bound threads. Change "bool mpsafe" to "int flags".
- Allow exit of LWPs in the IDL state when (l != curlwp).
- More locking fixes & conversion to the new API.
 1.337.2.2 10-Apr-2007  ad Sync with head.
 1.337.2.1 13-Mar-2007  ad Pull in the initial set of changes for the vmlocking branch.
 1.340.2.2 10-Sep-2007  skrll Sync with HEAD.
 1.340.2.1 15-Aug-2007  skrll Sync with HEAD.
 1.343.8.2 30-Jul-2007  taca Make it compile with WD_SOFTBADSECT option.
 1.343.8.1 30-Jul-2007  taca file wd.c was added on branch matt-mips64 on 2007-07-30 06:59:14 +0000
 1.343.6.4 23-Mar-2008  matt sync with HEAD
 1.343.6.3 09-Jan-2008  matt sync with HEAD
 1.343.6.2 08-Nov-2007  matt sync with -HEAD
 1.343.6.1 06-Nov-2007  matt sync with HEAD
 1.343.4.9 11-Dec-2007  jmcneill Wrong branch, backout previous.
 1.343.4.8 11-Dec-2007  jmcneill Disable the wd_suspend power handler.
 1.343.4.7 08-Dec-2007  jmcneill Rename pnp(9) -> pmf(9), as requested by many.
 1.343.4.6 11-Nov-2007  joerg Sync with HEAD.
 1.343.4.5 06-Nov-2007  joerg Refactor PNP API:
- Make suspend/resume directly a device functionality. It consists of
three layers (class logic, device logic, bus logic), all of them being
optional. This replaces D0/D3 transitions.
- device_is_active returns true if the device was not disabled and was
not suspended (even partially), device_is_enabled returns true if the
device was enabled.
- Change pnp_global_transition into pnp_system_suspend and
pnp_system_resume. Before running any suspend/resume handlers, check
that all currently attached devices support power management and bail
out otherwise. The latter is not done for the shutdown/panic case.
- Make the former bus-specific generic network handlers a class handler.
- Make PNP message like volume up/down/toogle PNP events. Each device
can register what events they are interested in and whether the handler
should be global or not.
- Introduce device_active API for devices to mark themselve in use from
either the system or the device. Use this to implement the idle handling
for audio and input devices. This is intended to replace most ad-hoc
watchdogs as well.
- Fix somes situations in which audio resume would lose mixer settings.
- Make USB host controllers better deal with suspend in the light of
shared interrupts.
- Flush filesystem cache on suspend.
- Flush disk caches on suspend. Put ATA disks into standby on suspend as
well.
- Adopt drivers to use the new PNP API.
- Fix a critical bug in the generic cardbus layer that made D0->D3
break.
- Fix ral(4) to set if_stop.
- Convert cbb(4) to the new PNP API.
- Apply the PCI Express SCI fix on resume again.
 1.343.4.4 31-Oct-2007  joerg Sync with HEAD.
 1.343.4.3 26-Oct-2007  joerg Sync with HEAD.

Follow the merge of pmap.c on i386 and amd64 and move
pmap_init_tmp_pgtbl into arch/x86/x86/pmap.c. Modify the ACPI wakeup
code to restore CR4 before jumping back into kernel space as the large
page option might cover that.
 1.343.4.2 16-Oct-2007  joerg Replace shutdown hook to flush cache with PNP style power handler to
flush the disk cache also on suspend.
 1.343.4.1 02-Oct-2007  joerg Sync with HEAD.
 1.345.2.1 14-Oct-2007  yamt sync with head.
 1.346.2.2 13-Nov-2007  bouyer Sync with HEAD
 1.346.2.1 25-Oct-2007  bouyer Sync with HEAD.
 1.348.2.3 18-Feb-2008  mjf Sync with HEAD.
 1.348.2.2 27-Dec-2007  mjf Sync with HEAD.
 1.348.2.1 19-Nov-2007  mjf Sync with HEAD.
 1.350.4.1 11-Dec-2007  yamt sync with head.
 1.350.2.2 26-Dec-2007  ad Sync with head.
 1.350.2.1 04-Dec-2007  ad Pull the vmlocking changes into a new branch.
 1.351.2.2 02-Jan-2008  bouyer Sync with HEAD
 1.351.2.1 13-Dec-2007  bouyer Sync with HEAD
 1.355.6.7 17-Jan-2009  mjf Sync with HEAD.
 1.355.6.6 29-Jun-2008  mjf Sync with HEAD.
 1.355.6.5 02-Jun-2008  mjf Sync with HEAD.
 1.355.6.4 06-Apr-2008  mjf - after some discussion with agc@ i agreed it would be a good idea to move
device_unregister_* to device_deregister_* to be more like the pmf(9)
functions, especially since a lot of the time the function calls are next
to each other.

- add device_register_name() support for dk(4).
 1.355.6.3 04-Apr-2008  mjf * Catch up with selnotify() changes.

* Catch up with some softc changes in HEAD.

* More s/dctl/devfsctl/ changes.

* Remove debugging printfs from init(8) that slipped in by mistake.

* Cosmetic changes in devfsd, allow multiple attributes to be set in
one rule. Switch from err() to syslog() messages.

* Fix etc/devfsd.conf, I'd made some changes previously that had syntax
errors.
 1.355.6.2 03-Apr-2008  mjf Sync with HEAD.
 1.355.6.1 29-Mar-2008  mjf - etc/devfsd.conf: Add some rules to give nodes like /dev/tty and
/dev/null better default modes, i.e. 0666.

- sbin/init: Run devfsd -s before going to multiuser.

- sys/arch: Provide arm32, i386, sparc with a mem_init() function to request
device nodes for /dev/null, /dev/zero, etc.

- sys/dev: Convert rnd, wd, agp, raid, cd, sd, wsdisplay, wskbd, wsmouse,
wsmux, tty, bpf, swap to devfs New World Order.

- sys/fs/devfs: Make the visibility attribute of device nodes configurable.
Also provide a function to mount a devfs on boot.

- sys/kern: Add a new boot flag, -n. This disables devfs support. Unless
the -n flag is specified the kernel will mount a devfs file
system on boot.
 1.355.2.1 24-Mar-2008  keiichi sync with head.
 1.360.4.5 11-Mar-2010  yamt sync with head
 1.360.4.4 20-Jun-2009  yamt sync with head
 1.360.4.3 16-May-2009  yamt sync with head
 1.360.4.2 04-May-2009  yamt sync with head.
 1.360.4.1 16-May-2008  yamt sync with head.
 1.360.2.2 17-Jun-2008  yamt sync with head.
 1.360.2.1 18-May-2008  yamt sync with head.
 1.362.2.1 23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.363.8.4 30-Jan-2010  snj Pull up following revision(s) (requested by bouyer in ticket #1269):
sys/dev/ata/wd.c: revision 1.383 via patch
struct buf::b_iodone is not called at splbio() any more.
Make sure non-MPsafe iodone callbacks raise the SPL as appropriate.
 1.363.8.3 10-Jan-2010  snj Pull up following revision(s) (requested by dsl in ticket #1220):
sys/dev/ata/wd.c: revision 1.380
sys/dev/ata/wdvar.h: revision 1.38
ATA sepcs changed the LBA48 boundary from 0xfffffff to 0xffffffe between
ATA6r1 and ATA6r3, which caused drives conform to ATA6r3 or later to
reject LBA28 commands at sector 0xfffffff.
Get the right idea from the LBA48 boundary from IDENTIFY words 60-61.
Remove the WD_QUIRK_FORCE_LBA48 quirk, associated tables entries and
autodetect code, it's not needed any more.
Based on patch sent to teck-kern by Christoph Badura, use of words 60-61
instead of a constant for the LBA48 boundary by me.
 1.363.8.2 10-Jan-2010  snj Pull up following revision(s) (requested by dsl in ticket #1231):
sys/dev/ata/atareg.h: revision 1.33
sys/dev/ata/wd.c: revision 1.378
Add atap_max_lba member to ataparams for LBA48 Maximum Address. From OpenBSD.
While here, renumber some __reserved member names to be sequential.
Also, add a member for the World Wide Name.
Use atap_max_lba in wd.c instead of offsets into a __reserved member.
 1.363.8.1 18-Feb-2009  snj branches: 1.363.8.1.2; 1.363.8.1.4;
Pull up following revision(s) (requested by drochner in ticket #441):
sys/dev/ata/wd.c: revision 1.368
sys/dev/pcmcia/wdc_pcmcia.c: revision 1.113
-in the wdc@pcmcia driver, don't try to powerup/down while cold or
shutting down -- this is only supposed to happen from the pcmcia
kernel thread while the system is running
-in wd_shutdown, call ata_addref first, so that the adapter is either
active or we know that it isn't
This fixes a hang on shutdown-p reported by Dennis den Brok per
PR kern/40531. Verified by by Dennis and me.
(reported against 5.0-Beta, so this is a pullup candidate)
 1.363.8.1.4.1 21-Apr-2010  matt sync to netbsd-5
 1.363.8.1.2.1 30-Jan-2010  snj Pull up following revision(s) (requested by bouyer in ticket #1269):
sys/dev/ata/wd.c: revision 1.383 via patch
struct buf::b_iodone is not called at splbio() any more.
Make sure non-MPsafe iodone callbacks raise the SPL as appropriate.
 1.363.6.3 28-Apr-2009  skrll Sync with HEAD.
 1.363.6.2 03-Mar-2009  skrll Sync with HEAD.
 1.363.6.1 19-Jan-2009  skrll Sync with HEAD.
 1.363.4.1 13-Dec-2008  haad Update haad-dm branch to haad-dm-base2.
 1.368.2.2 23-Jul-2009  jym Sync with HEAD.
 1.368.2.1 13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.383.2.2 06-Nov-2010  uebayasi Sync with HEAD.
 1.383.2.1 30-Apr-2010  uebayasi Sync with HEAD.
 1.384.2.1 05-Mar-2011  rmind sync with head
 1.385.4.1 17-Feb-2011  bouyer Sync with HEAD
 1.385.2.1 06-Jun-2011  jruoho Sync with HEAD.
 1.389.2.4 22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.389.2.3 23-Jan-2013  yamt sync with head
 1.389.2.2 30-Oct-2012  yamt sync with head
 1.389.2.1 17-Apr-2012  yamt sync with head
 1.390.2.1 18-Feb-2012  mrg merge to -current.
 1.392.2.1 03-Jul-2012  jdc Pull up revision 1.393 (requested by bouyer in ticket #381).

In some case, when an error is reported by the disk, the ahci controller
still reports a number of bytes transfered equal to bcount.
This then triggers a KASSERT in physio_biodone:
if (done == todo)
KASSERT(bp->b_error == 0);
Detect this case in wd(4) (so that the workaround works for other controllers
too if they have the same issue, or if the issue is with the drive)
and claim we didn't read/write anything.
 1.400.2.6 03-Dec-2017  jdolecek update from HEAD
 1.400.2.5 20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.400.2.4 23-Jun-2013  tls resync from head
 1.400.2.3 25-Feb-2013  tls resync with head
 1.400.2.2 20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.400.2.1 09-Oct-2012  bouyer Support transfers of up to MACHINE_MAXPHYS in all pciide variants, and ahci.
wd(4) limits its maxphys depending on the drives's capability (64k sectors
for LBA48, 256 sectors for LBA and 128 sectors for older devices).

I assumed all pciide controllers could do MACHINE_MAXPHYS transfers, but
this may not be true. The capabilities of each controller variants should be
looked at more closely.
 1.403.2.2 18-May-2014  rmind sync with head
 1.403.2.1 28-Aug-2013  rmind sync with head
 1.407.2.2 10-Aug-2014  tls Rebase.
 1.407.2.1 07-Apr-2014  tls Be a little more clear and consistent about harvesting entropy from devices:

1) deprecate RND_FLAG_NO_ESTIMATE

2) define RND_FLAG_COLLECT_TIME, RND_FLAG_COLLECT_VALUE

3) define RND_FLAG_ESTIMATE_TIME, RND_FLAG_ESTIMATE_VALUE

4) define RND_FLAG_DEFAULT: RND_FLAG_COLLECT_TIME|
RND_FLAG_COLLECT_VALUE|RND_FLAG_ESTIMATE_TIME

5) Make entropy harvesting from environmental sensors a little more generic
and remove it from individual sensor drivers.

6) Remove individual open-coded delta-estimators for values from a few
places in the tree (uvm, environmental drivers).

7) 0 -> RND_FLAG_DEFAULT, actually gather entropy from various drivers
that had stubbed out code, other minor cleanups.
 1.412.2.2 05-Jul-2016  snj Pull up following revision(s) (requested by bouyer in ticket #1186):
sys/dev/ata/wd.c: revision 1.421
sys/kern/subr_autoconf.c: revision 1.242 via patch
sys/sys/device.h: revision 1.149
Add a new config_detach() flag, DETACH_POWEROFF, which is set when
detaching devices at shutdown time with RB_POWERDOWN.
When detaching wd(4), put the drive in standby before detach
for DETACH_POWEROFF.
Fix PR kern/51252
 1.412.2.1 11-Nov-2014  martin Pull up following revision(s) (requested by mlelstv in ticket #201):
sbin/dkctl/dkctl.8: revision 1.24
sbin/dkctl/dkctl.8: revision 1.25
sys/dev/scsipi/sd.c: revision 1.310
sys/dev/ata/wd.c: revision 1.415
sbin/dkctl/dkctl.c: revision 1.21
sys/dev/raidframe/rf_netbsdkintf.c: revision 1.315
sys/dev/ld.c: revision 1.78
sys/dev/vnd.c: revision 1.234
sys/dev/dksubr.c: revision 1.54
sys/sys/dkio.h: revision 1.20
sys/dev/dkwedge/dk.c: revision 1.74
Add ioctl to autodiscover wedges.
Implement DIOCMWEDGES ioctl that triggers wedge autodiscovery.
Also fix a reference counting bug and clean up some code.
support DIOCMWEDGES ioctl.
Add 'makewedges' option to autodiscover wedges from a changed label.
New sentence, new line. Bump date for previous.
 1.415.2.6 28-Aug-2017  skrll Sync with HEAD
 1.415.2.5 05-Dec-2016  skrll Sync with HEAD
 1.415.2.4 05-Oct-2016  skrll Sync with HEAD
 1.415.2.3 09-Jul-2016  skrll Sync with HEAD
 1.415.2.2 06-Jun-2015  skrll Sync with HEAD
 1.415.2.1 06-Apr-2015  skrll Sync with HEAD
 1.421.2.5 26-Apr-2017  pgoyette Sync with HEAD
 1.421.2.4 20-Mar-2017  pgoyette Sync with HEAD
 1.421.2.3 07-Jan-2017  pgoyette Sync with HEAD. (Note that most of these changes are simply $NetBSD$
tag issues.)
 1.421.2.2 06-Aug-2016  pgoyette Sync with HEAD
 1.421.2.1 26-Jul-2016  pgoyette Sync with HEAD
 1.427.2.1 21-Apr-2017  bouyer Sync with HEAD
 1.428.2.36 07-Oct-2017  jdolecek make usage of NCQ 'high' priority for BPRIO_TIMECRITICAL xfers settable via
sysctl, too
 1.428.2.35 28-Sep-2017  jdolecek add sysctls to control if NCQ is being used, and how many max tags;
I have a drive which is significantly slower with NCQ than non-NCQ,
and it's generally useful to have this easily overridable

while here, also move the frequency settings for WD_CHAOS_MONKEY
to a sysctl and make it per-drive
 1.428.2.34 13-Aug-2017  jdolecek by default make the chaos monkey do nothing, so it's easier to have this
compiled in all the time for testing; the vars can be set via DDB
when needed
 1.428.2.33 13-Aug-2017  jdolecek restore the fallback to non-NCQ on retries, do this after WDIORETRIES_SINGLE
retries, but only for non-FUA I/O; also only do the ATA_SINGLE fallback
when non-FUA

this makes sure that bio with ATA_SINGLE is not attempted as NCQ -
the ATA_SINGLE I/O is usually done using PIO by drivers which
actually support it, and thus are not compatible with DMA-only NCQ
 1.428.2.32 12-Aug-2017  jdolecek do not reset drive after successful NCQ error recovery
 1.428.2.31 30-Jul-2017  jdolecek actually change the retry to also use NCQ - with one drive I see
very frequent fatal errors on siisata when switching often between
NCQ and non-NCQ I/O commands with chaos monkey (basically, always
the next NCQ command on a slot which had non-NCQ I/O throws fatal
error); they completely vanish when just using NCQ all the time
 1.428.2.30 29-Jul-2017  jdolecek actually count the REQUEUE as retry also, so that it will be retried
as non-NCQ, will not be subject to chaos monkey, and reported as fixed
once finished, too
 1.428.2.29 23-Jul-2017  jdolecek for wd, only call disk_busy() on the first try, do not call it on retries,
as unbusy is called just once when the xfer is finished

also noticed in PR kern/43169 by Matthias Pfaller, but contrary to suggested
fix done in way to keep the disk marked busy during the timeouts, as I think
it's more correct
 1.428.2.28 21-Jul-2017  jdolecek KASSERT() that chaosed xfer actually ends up with error; might end up being
returned as successful due to bugs in error recovery code
 1.428.2.27 19-Jul-2017  jdolecek defflag WD_CHAOS_MONKEY into opt_wd.h together with WD_SOFTBADSECT to set/unset
this more easily
 1.428.2.26 19-Jul-2017  jdolecek tighen and expand error handling, mostly for NCQ use cases:
- make retry timeout callout per xfer, i.e. retry separately
- zero whole bio struct on retry to avoid more stale state
- add a REQUEUE option, which doesn't bump retry count
- add ata_read_log_ext_ncq() for NCQ recovery
- adjust logic for activating xfers - allow next command only when
it's for same drive, several concurrent are only supported when HBA
and driver support FIS-based switching
- add new ata_timeout() which handles race between callout_stop()
and the invokation, add appropriate handling on deactivate/free paths
- stop using ch_status/ch_error in non-wdc code; later it will be dropped
completely
 1.428.2.25 03-Jul-2017  jdolecek reset xfer c_flags before retry, to clear flags like C_TIMEOU, or C_NCQ,
so that retry, and no-NCQ downgrade logic actually works - drivers
typically doesn't reset this field

print number of retries to make it easier to spot the same xfer being
retried several times

in wddone(), hold the wd lock only when reading/changing wd softc
structures, and not e.g. when calling malloc(), rnd_add_uint32() or
ata_free_xfer(), which have their own locks; initially done to fix
diagnostic assertion about held spin lock in kpause() within
ata_reset_drive hook, but need to run that hook with AT_POLL anyway,
since wddone() is typically invoked from interrupt context

fix another interrupt context bug for WD_SOFTBADSECT - the malloc() needs
to be called with M_NOWAIT
 1.428.2.24 03-Jul-2017  jdolecek introduce some code to test retry paths
 1.428.2.23 24-Jun-2017  jdolecek only limit the openings for I/O xfer if the drive actually supports NCQ; if
it's non-NCQ drive, the tag is not going to be used, so we can use any xfer
 1.428.2.22 23-Jun-2017  jdolecek add debug printf if we fail to obtain xfer in wddump()
 1.428.2.21 23-Jun-2017  jdolecek change wd_standby() to remove the ata_channel_start() call - the purpose
of that particular command is to make the drive idle

fix locking bug in wddetach() exposed by calling ata_channel_start()
in wd_standby() - move wd_standby() call out of the section protected
by drive mutex, to avoid lock against itself should it need to get
the lock

change wd_flushcache() to only call ata_channel_start() when called from
the ioctl; particularly, don't call it when suspending, closing, or on shutdown
 1.428.2.20 23-Jun-2017  jdolecek restart I/O processing after freeing xfer, i.e. now even after commands
like cache flush or standby; the command handling no longer use on-stack xfer,
hence use queue slot and compete with normal I/O for the xfers

the restart give change to all drives attached to the same channel
in round-robin fashion, for fair usage and to recover from situation
when disk is idle due to all xfers being consumed by other drives

make special concession for flush cache - ignore any new I/O requests
on the particular disk while the flush cache is waiting for xfer,
so that I/O queue won't starve the flush cache and the flush cache would
be done ASAP

tested on piixide(4), ahci(4), siisata(4)
 1.428.2.19 20-Jun-2017  jdolecek remove ata queue downsizing - every device, attached to the same channel,
uses slots according to it's own limits

wdc code changed to expect maximum one active xfer, and not check number
of openings in the channel; this is to facilitate using wdc functions
for e.g. handling of atapi commands for drivers which support both ATAPI
and NCQ
 1.428.2.18 19-Jun-2017  jdolecek add ata_channel lock, use it to protect queue manipulation (only that for now);
add ata_channel_detach() to destroy the locks

change ata_get_xfer() so that it can wait for xfer, convert all on-stack
xfer code to use the blocking variant

fix siisata_reset_drive() to use polled reset and not try ata_activate_xfer(),
convert drive probe code also over from slot0 XXX to ata_get_xfer()

drive reset and PMP now works on siisata(4) too; changes tested also
on piixide(4), ahci(4), mvsata(4)
 1.428.2.17 19-Jun-2017  jdolecek ata subsystem needs more work before wd(4) could be MPSAFE
 1.428.2.16 16-Jun-2017  jdolecek adjust reset channel and dump paths
- channel reset now always kills active transfer, even on dump path, but
now doesn't touch the queued waiting transfers; also kill_xfer hook is
always called, so that HBA can free any private xfer resources and thus
the dump request has chance to work
- kill_xfer routines now always call ata_deactivate_xfer(); added KASSERT()s
to ata_free_xfer() to expect deactivated xfer
- when called during channel reset before dump, ata_kill_active() drops
any queued waiting transfers without processing
- do not (re)queue any transfers in wddone() when dumping
- kill AT_RST_NOCMD flag

This should also hopefully fix the 'polled command has been queued' panic
as reported in:
PR kern/11811 by John Hawkinson
PR kern/47041 by Taylor R Campbell
PR kern/51979 by Martin Husemann

dump tested working with piixide(4) and ahci(4). mvsata(4) dump times out,
but otherwise tested working, will be fixed separately. siisata(4) mechanically
changed and not tested.
 1.428.2.15 14-Jun-2017  jdolecek put back reset of retry count to 0 when starting processing a buf

do not use NCQ for retried transfers to speed up recovery, unless FUA
is requested
 1.428.2.14 24-Apr-2017  jdolecek set NCQ priority field to 'high' for BPRIO_TIMECRITICAL transfers if drive
supports it
 1.428.2.13 24-Apr-2017  jdolecek ata_free_xfer() must be called with lock held, same as ata_get_xfer()
 1.428.2.12 24-Apr-2017  jdolecek sync with HEAD, remove the nonfunctional Seagate 'mod15write' bug workaround
 1.428.2.11 23-Apr-2017  jakllsch Simplify wi_find() with container_of().

No need to search a list of wd_ioctls when the *bp is always going to be within the wd_ioctl structure.

Correctly initialize wi->wi_bp.b_dev in the non-data case anyway.

Fixes ATAIOCCOMMAND procesing on driver instances other than wd0.
 1.428.2.10 22-Apr-2017  jakllsch Avoid destroying uninitialized xfer on error in wdioctlstrategy().
 1.428.2.9 19-Apr-2017  jdolecek add ATA FUA support
 1.428.2.8 19-Apr-2017  jdolecek up to 65536 sectors can be transferred for LBA48 commands; adjust wdminphys()
and wdstart1() accordingly
 1.428.2.7 19-Apr-2017  jdolecek adjust ata code to support more than one active command, including the
timeout handling, add support for NCQ commands

move probe for NCQ and number of tags to middle layer, negotiate mutual
support between drive and controller

implement NCQ support in ahci(4)
 1.428.2.6 15-Apr-2017  jdolecek tweak english
 1.428.2.5 15-Apr-2017  jdolecek probe for NCQ and WRITE DMA FUA EXT support
 1.428.2.4 15-Apr-2017  jdolecek pass also ata_command via ata_xfer, callers of ata_exec_command() is now
responsible for allocation/disposal of the structure

change code to allocate ata_xfer for commands on stack same way as previously
the ata_command were, using c_slot 0; adjust asserts so that it would allow
several xfers with same c_slot, as long as only one such transfer is active
at a time
 1.428.2.3 15-Apr-2017  jdolecek make ata_xfer's allocated as part of ata_queue and make it include ata_bio;
they are pre-allocated on attach and ata_get_xfer() now never sleep, drop the
pool

modify wd(4) to file the bio requests using the xfers and hence
make it possible to have more than one active I/O request in flight;
ata_bio callback doesn't need to allocate any memory any more,
require it to never return ATACMD_TRY_AGAIN

move lp, badsect, multi from ata_bio to ata_drive_datas, as they are per-drive,
not per transfer

drop unused drv atac_claim_hw/atac_free_hw hooks, and also drop again ata_bio
c_hwslot
 1.428.2.2 12-Apr-2017  jdolecek more MP
 1.428.2.1 12-Apr-2017  jdolecek convert to mutex, mark MPSAFE
 1.429.2.1 30-Apr-2017  pgoyette And a few more DEVSW_MODULE_INIT for modular drivers
 1.438.2.3 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.438.2.2 06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.438.2.1 25-Jun-2018  pgoyette Sync with HEAD
 1.439.2.4 21-Apr-2020  martin Sync with HEAD
 1.439.2.3 13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.439.2.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.439.2.1 10-Jun-2019  christos Sync with HEAD
 1.441.2.12 06-Oct-2018  jdolecek actually, just make dump use the same queue skip as recovery, and remove the
no longer necessary ata_queue_reset() call from wd(4)

also for PR kern/47041
 1.441.2.11 06-Oct-2018  jdolecek remove AT_RST_EMERG, do the queue reset explicitly in wd(4)

this should explicitly fix PR kern/47041 with sync during heavy
disk activity, even thought it was actually already implicitly fixed by calling
ata_thread_run() for drive reset in previous commit already, since the
function already called ata_queue_reset()

drop now unused ch_reset_flags and drive_reset_flags
 1.441.2.10 06-Oct-2018  jdolecek fix dump to also hold channel lock for the drive reset
 1.441.2.9 04-Oct-2018  jdolecek relax the NOERROR + C_CHAOS, just rerun the xfer instead of panic, to handle
another variant of bad/missing AHCI error recovery, this time under Parallels
 1.441.2.8 04-Oct-2018  jdolecek don't add +1 to retry on error
 1.441.2.7 03-Oct-2018  jdolecek change channel reset and drive reset for all ATA controllers to always
run via thread, and with channel lock held the whole time; the queue is
frozen while reset is pending

for this repurpose ata_reset_channel() into new ata_thread_run()

also adjust some device printfs to not leak xfer pointer, and avoid
aprint_* for non-autoconf messages
 1.441.2.6 22-Sep-2018  jdolecek remove explicit ata_channel_start() calls, it's no longer necessary
now that ata_xfer's are allocated via pool and not really limited;
replace by just a callout to restart the processing for rare cases
where system runs out of memory
 1.441.2.5 22-Sep-2018  jdolecek fix use-after-free in wd(4) dump, detected by switch to the pool

change code in wd_dumpblocks() to use it's own non-pool ata_xfer,
which skips the deallocation step and thus keeps the contents when the I/O
is finished
 1.441.2.4 22-Sep-2018  jdolecek separate ata_xfer slot allocation and the memory allocation, so that
there can be more queued xfers than number of supported slots by controller,
and use a pool instead of custom pre-allocation

primarily to help PR kern/52614

remove no longer needed custom wd(4) logic for flush cache

switch also wd(4) trim/suspend/setcache/wdioctlstrategy to sleep waiting
for the memory, they are all called from process context and this
avoids spurious failures
 1.441.2.3 17-Sep-2018  jdolecek convert from malloc()/free() to kmem_zalloc()/kmem_free()
 1.441.2.2 01-Sep-2018  jdolecek stop assuming c_slot is immutable, use the xfer pointer rather than c_slot
in printfs()
 1.441.2.1 31-Aug-2018  jdolecek refactor ata_xfer to be just dumb structure; move all callouts/condvars out

retry callout to wd(4); reset callout and the active/cmd finish condvars
to channel queue; change code using the condvars so it works if there
are multiple waiters

simplify the async wait code for cmds, replace ata_wait_xfer()/ata_wake_xfer()
with ata_wait_cmd()

fix the callout_invoking/ack race handling code for timeouts to
actually have chance to work; change mvsata(4) to use generic timeout func

towards resolution of kern/52614
 1.452.2.3 30-Dec-2022  martin Pull up following revision(s) (requested by tsutsui in ticket #1557):

sys/dev/ic/ahcisata_core.c: revision 1.83
sys/dev/ic/ahcisata_core.c: revision 1.102
sys/dev/ata/ata.c: revision 1.164
sys/dev/ata/ata_wdc.c: revision 1.115
sys/dev/ata/ata_recovery.c: revision 1.4
sys/dev/ic/siisata.c: revision 1.42
sys/dev/ic/wdc.c: revision 1.308
sys/dev/ic/mvsata.c: revision 1.56
sys/dev/scsipi/atapi_wdc.c: revision 1.138
sys/dev/ic/siisata.c: revision 1.49
sys/dev/ata/atavar.h: revision 1.105
sys/dev/ata/wd.c: revision 1.460
sys/dev/ata/ata.c: revision 1.155
sys/dev/ata/wd.c: revision 1.462
sys/dev/ata/atavar.h: revision 1.109
sys/dev/ata/satapmp_subr.c: revision 1.16
sys/dev/ic/wdc.c: revision 1.299
sys/dev/ic/ahcisata_core.c: revision 1.93
sys/dev/ata/ata_wdc.c: revision 1.120
sys/dev/ic/wdcvar.h: revision 1.100
sys/dev/scsipi/atapi_wdc.c: revision 1.141
sys/dev/ic/mvsata.c: revision 1.61
sys/dev/usb/umass_isdata.c (apply patch)

drop wd lock in wdstart1() before calling the ata_bio hook; when called
from ata thread context, that can still need to sleep for wdc attachments
in wdcwait()

fix use-after-free for ata xfer on bio submission found by KASAN
driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself
PR kern/55169 by Nick Hudson

Function declaration formating whitespace consistency. NFCI.

PR kern/56403
Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.
(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.
Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().
(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).
"Go ahead" by jdolecek@.
 1.452.2.2 21-Mar-2020  martin Pull up following revision(s) (requested by riastradh in ticket #795):

sys/dev/dksubr.c: revision 1.112
sys/arch/xen/xen/xbd_xenbus.c: revision 1.95
sys/dev/scsipi/sd.c: revision 1.328
sys/dev/dkvar.h: revision 1.32
sys/dev/ld.c: revision 1.109
sys/dev/cgd.c: revision 1.120
sys/dev/raidframe/rf_netbsdkintf.c: revision 1.380
sys/dev/ata/wd.c: revision 1.458

Add a flag to dk_dump for virtual disk devices.

If a disk is backed by a physical medium other than itself, such as
cgd(4), then it passes DK_DUMP_RECURSIVE to disable the recursion
detection for dk_dump.

If, however, a device represents a physical medium on its own, such
as wd(4), then it passes 0 instead.

With this, I can now dump to dk on cgd on dk on wd.
 1.452.2.1 21-Mar-2020  martin Pull up following revision(s) (requested by riastradh in ticket #788):

sys/sys/dkio.h: revision 1.26
sys/dev/dkwedge/dk.c: revision 1.100
sys/sys/disk.h: revision 1.75
external/cddl/osnet/dist/uts/common/fs/zfs/vdev_disk.c: revision 1.14
external/cddl/osnet/dist/uts/common/fs/zfs/vdev_disk.c: revision 1.15
sys/dev/cgd.c: revision 1.121
sys/dev/ata/wdvar.h: revision 1.50
sys/kern/subr_disk_open.c: revision 1.15
sys/dev/ata/wd.c: revision 1.459

New ioctl DIOCGSECTORALIGN returns sector alignment parameters.

struct disk_sectoralign {
/* First aligned sector number. */
uint32_t dsa_firstaligned;
/* Number of sectors per aligned unit. */
uint32_t dsa_alignment;
};

- Teach wd(4) to get it from ATA.
- Teach cgd(4) to pass it through from the underlying disk.
- Teach dk(4) to pass it through with adjustments.
- Teach zpool (zfs) to take advantage of it.
=> XXX zpool doesn't seem to understand when the vdev's starting
sector is misaligned.

Missing:
- ccd(4) and raidframe(4) support -- these should support _using_
DIOCGSECTORALIGN to decide where to start putting ccd or raid
stripes on disk, and these should perhaps _implement_
DIOCGSECTORALIGN by reporting the stripe/interleave factor.
- sd(4) support -- I don't know any obvious way to get it from SCSI,
but if any SCSI wizards know better than I, please feel free to
teach sd(4) about it!
- any ld(4) attachments -- might be worth teaching the ld drivers for
nvme and various raid controllers to get the aligned sector size

There's some duplicate logic here for now. I'm doing it this way,
rather than gathering the logic into a new disklabel_sectoralign
function or something, so that this change is limited to adding a new
ioctl, without any new kernel symbols, in order to make it easy to
pull up to netbsd-9 without worrying about the module ABI.

Make getdiskinfo() compatible with a DIOCGWEDGEINFO.

dkw_parent is defined to hold the disk name as used by disk_find(), not
a partition (i.e. no partition letter appended).

Use utility functions to handle disk geometry.
 1.453.2.2 25-Jan-2020  ad Sync with head.
 1.453.2.1 17-Jan-2020  ad Sync with head.
 1.460.2.1 20-Apr-2020  bouyer Sync with HEAD
 1.469.2.1 02-Aug-2025  perseant Sync with HEAD
 1.9 12-Oct-1998  bouyer Merge bouyer-ide
 1.8 15-Aug-1998  mycroft Assign my copyrights to TNF.
 1.7 26-Apr-1998  mycroft Update copyright.
 1.6 26-Apr-1998  mycroft In LBA mode, always use the `total capacity' count, rather than calculating
the capacity based on the c/h/s numbers. In fact, don't use the c/h/s
numbers for much of anything.
For ATA-4 drives or later, always use LBA mode, since it's now required.

Collectively, this allows >8GB disks (like the 12GB Bigfoot) to work.
 1.5 23-Apr-1998  bouyer From Robert V. Baron:
Compute the disk block addr at command queing time rather than exec time.
This fix a bug which could lead to data corruption on disk: when a command
was reexecuted after an error condition (from wdcunwedge), the partition
offset was re-added to the block addr, leading to a transfert at the wrong
disk block.
This should fix the problem reported by some laptop's users, where the
first disk read after a suspend/resume returned garbage.
 1.4 14-Jan-1998  cgd Various cleanups and bug fixes to the wdc/wd code:
* Fix bug in wdc that would overflow ATAPI transfer length.
* Improve wdc probe code so that 'wdc' is probed in if present
even if there are no drives attached, and so that it works
properly even if the only device is an ATAPI slave.
* bus_space-ify.
* split the ISA attachment from the wdc driver, and remove
ISA dependencies from non-ISA files.
* claim that wd and wdc are now machine-independent (probably not
completely true, but mostly so; they at least work on arm32 and
i386).
* Various other minor fixups and cleanups, some of which were pointed
out by Kazuki Sakamoto.
 1.3 03-Sep-1997  bouyer Add a map to wdc_softc to track the already known devices to prevent them to
be probed several times. This fixes the "ATAPI CD probed as wd drive" problem.
Thanks to Geoff Wing <mason@primenet.com.au> for testing this on his hardware.
 1.2 27-Aug-1997  bouyer branches: 1.2.2;
Merge scsipi branch in the mainline. This add support for ATAPI devices
(currently only CD-ROM drives on i386). The sys/dev/scsipi system provides 2
busses to which devices can attach (scsibus and atapibus). This needed to
change some include files and structure names in the low level scsi drivers.
 1.1 01-Jul-1997  bouyer branches: 1.1.2;
file wdlink.h was initially added on branch bouyer-scsipi.
 1.1.2.1 01-Jul-1997  bouyer Add the ide atapi driver from my original atapi driver, ported
to scsipi.
 1.2.2.2 04-Sep-1997  thorpej Update marc-pcmcia branch from trunk.
 1.2.2.1 27-Aug-1997  thorpej Update marc-pcmcia branch from trunk.
 1.52 13-Apr-2025  jakllsch Add physical sector and alignment info to struct disk_geom and the
geometry plist, and handle in partutil.

Bump version for disk_geom addition.

Collect DIOCGSECTORALIGN handling into one place.
 1.51 28-Dec-2021  riastradh branches: 1.51.10;
wd(4): Fix bugs in softbadsect handling.

- Don't copyout kernel virtual addresses (of SLIST entries) that
userland won't use anyway.
=> The structure still has space for this pointer; it's just always
null when userland gets it now.

- Don't copyout under a lock.

- Stop and return error if copyout fails (unless we've already copied
some out).

- Don't kmem_free under a lock.

XXX Unclear whether anyone actually uses WD_SOFTBADSECT or why --
it's always been disabled by default. Maybe we should just remove
it?
 1.50 02-Mar-2020  riastradh New ioctl DIOCGSECTORALIGN returns sector alignment parameters.

struct disk_sectoralign {
/* First aligned sector number. */
uint32_t dsa_firstaligned;

/* Number of sectors per aligned unit. */
uint32_t dsa_alignment;
};

- Teach wd(4) to get it from ATA.
- Teach cgd(4) to pass it through from the underlying disk.
- Teach dk(4) to pass it through with adjustments.
- Teach zpool (zfs) to take advantage of it.
=> XXX zpool doesn't seem to understand when the vdev's starting
sector is misaligned.

Missing:

- ccd(4) and raidframe(4) support -- these should support _using_
DIOCGSECTORALIGN to decide where to start putting ccd or raid
stripes on disk, and these should perhaps _implement_
DIOCGSECTORALIGN by reporting the stripe/interleave factor.

- sd(4) support -- I don't know any obvious way to get it from SCSI,
but if any SCSI wizards know better than I, please feel free to
teach sd(4) about it!

- any ld(4) attachments -- might be worth teaching the ld drivers for
nvme and various raid controllers to get the aligned sector size

There's some duplicate logic here for now. I'm doing it this way,
rather than gathering the logic into a new disklabel_sectoralign
function or something, so that this change is limited to adding a new
ioctl, without any new kernel symbols, in order to make it easy to
pull up to netbsd-9 without worrying about the module ABI.
 1.49 05-Apr-2019  bouyer branches: 1.49.4;
Implement a DIRTY flag (copied from sd(4)) so avoid flushing the cache if
there has been no write. This avoids a (long) timeout on the flush cache
command triggered by atactl sleep, when the device is open only by the atactl
command itself.
If a drive has no partition open and goes to sleep, the WDF_LOADED
flag is clear, and the next open will issue wd_get_params() command.
But to wake up the drive a reset is required, and wd_get_params() doens't
issue a reset on timeout, so there's no way to wake up the disk.
Add a retry after reset to wd_get_params().

Tested by Hauke Fath; fixes PR kern/49457
 1.48 19-Mar-2019  mlelstv Set disk model name as disk type. The information can be queried through
drvctl(4).
 1.47 22-Oct-2018  jdolecek Merge jdolecek-ncqfixes branch

- ata_xfer's are dynamicall allocated as needed using a pool, no longer
limited to number of possible openings supported by controller; dump
and recovery paths use dedicated pre-allocated storage
- moved callouts and condvars from ata_xfer to queue or channel, so that
ata_xfer does not need special initialization
- slot allocation now done when xfer is being activated, uncoupled
from memory allocation; active slots are no longer tracked by controller
code
- channel and drive reset is done always via the atabus thread, and
now executes with channel locked the whole time
- NCQ recovery moved to shared function, and run via the thread also
- added some workarounds for buggy error recovery AHCI emulation in QEMU
and Parallels

designed to primarily fix kern/52614, but might also help with kern/47041
and kern/53183
 1.46 03-Nov-2017  mlelstv branches: 1.46.2; 1.46.4; 1.46.6;
Add WDF_OPEN flag to really disallow opening of a disk that has been invalidated.

Restore wdbiorestart function to actually retry the failed I/O request instead
of just restarting the queue.

Fix compilation without ATADEBUG.
 1.45 01-Nov-2017  mlelstv refactor wd and ataraid drivers to use common disk subroutines.
 1.44 07-Oct-2017  jdolecek Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.43 22-Jul-2016  jakllsch branches: 1.43.4;
Add ATA8-ACS Long Logical Sector Feature Set support to wd(4).
 1.42 13-Apr-2015  riastradh branches: 1.42.2;
Convert sys/dev to use <sys/rndsource.h>.
 1.41 10-Sep-2014  matt branches: 1.41.2;
g/u_int[0-9]*_t/ s/u_int/uint/g
 1.40 02-Feb-2012  tls branches: 1.40.6;
Entropy-pool implementation move and cleanup.

1) Move core entropy-pool code and source/sink/sample management code
to sys/kern from sys/dev.

2) Remove use of NRND as test for presence of entropy-pool code throughout
source tree.

3) Remove use of RND_ENABLED in device drivers as microoptimization to
avoid expensive operations on disabled entropy sources; make the
rnd_add calls do this directly so all callers benefit.

4) Fix bug in recent rnd_add_data()/rnd_add_uint32() changes that might
have lead to slight entropy overestimation for some sources.

5) Add new source types for environmental sensors, power sensors, VM
system events, and skew between clocks, with a sample implementation
for each.

ok releng to go in before the branch due to the difficulty of later
pullup (widespread #ifdef removal and moved files). Tested with release
builds on amd64 and evbarm and live testing on amd64.
 1.39 19-Nov-2011  tls branches: 1.39.2;
First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>. This change includes
the following:

An initial cleanup and minor reorganization of the entropy pool
code in sys/dev/rnd.c and sys/dev/rndpool.c. Several bugs are
fixed. Some effort is made to accumulate entropy more quickly at
boot time.

A generic interface, "rndsink", is added, for stream generators to
request that they be re-keyed with good quality entropy from the pool
as soon as it is available.

The arc4random()/arc4randbytes() implementation in libkern is
adjusted to use the rndsink interface for rekeying, which helps
address the problem of low-quality keys at boot time.

An implementation of the FIPS 140-2 statistical tests for random
number generator quality is provided (libkern/rngtest.c). This
is based on Greg Rose's implementation from Qualcomm.

A new random stream generator, nist_ctr_drbg, is provided. It is
based on an implementation of the NIST SP800-90 CTR_DRBG by
Henric Jungheim. This generator users AES in a modified counter
mode to generate a backtracking-resistant random stream.

An abstraction layer, "cprng", is provided for in-kernel consumers
of randomness. The arc4random/arc4randbytes API is deprecated for
in-kernel use. It is replaced by "cprng_strong". The current
cprng_fast implementation wraps the existing arc4random
implementation. The current cprng_strong implementation wraps the
new CTR_DRBG implementation. Both interfaces are rekeyed from
the entropy pool automatically at intervals justifiable from best
current cryptographic practice.

In some quick tests, cprng_fast() is about the same speed as
the old arc4randbytes(), and cprng_strong() is about 20% faster
than rnd_extract_data(). Performance is expected to improve.

The AES code in src/crypto/rijndael is no longer an optional
kernel component, as it is required by cprng_strong, which is
not an optional kernel component.

The entropy pool output is subjected to the rngtest tests at
startup time; if it fails, the system will reboot. There is
approximately a 3/10000 chance of a false positive from these
tests. Entropy pool _input_ from hardware random numbers is
subjected to the rngtest tests at attach time, as well as the
FIPS continuous-output test, to detect bad or stuck hardware
RNGs; if any are detected, they are detached, but the system
continues to run.

A problem with rndctl(8) is fixed -- datastructures with
pointers in arrays are no longer passed to userspace (this
was not a security problem, but rather a major issue for
compat32). A new kernel will require a new rndctl.

The sysctl kern.arandom() and kern.urandom() nodes are hooked
up to the new generators, but the /dev/*random pseudodevices
are not, yet.

Manual pages for the new kernel interfaces are forthcoming.
 1.38 17-Dec-2009  bouyer branches: 1.38.12;
ATA sepcs changed the LBA48 boundary from 0xfffffff to 0xffffffe between
ATA6r1 and ATA6r3, which caused drives conform to ATA6r3 or later to
reject LBA28 commands at sector 0xfffffff.
Get the right idea from the LBA48 boundary from IDENTIFY words 60-61.
Remove the WD_QUIRK_FORCE_LBA48 quirk, associated tables entries and
autodetect code, it's not needed any more.
Based on patch sent to teck-kern by Christoph Badura, use of words 60-61
instead of a constant for the LBA48 boundary by me.
 1.37 19-Oct-2009  bouyer Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !
 1.36 18-Mar-2008  cube branches: 1.36.4; 1.36.14; 1.36.22;
Split device_t and softc for ATA devices, as well as wd(4). Other
cosmetic changes where appropriate.
 1.35 28-Feb-2008  drochner use pmf again to register the shutdown handler, instead of
shutdownhook_establish()
 1.34 22-Feb-2008  dogcow As pointed out by Geoff Wing, 1.356 of wd.c needs sc_sdhook restored.
 1.33 09-Dec-2007  jmcneill branches: 1.33.6; 1.33.10;
Merge jmcneill-pm branch.
 1.32 11-Dec-2005  christos branches: 1.32.44; 1.32.46; 1.32.56; 1.32.58;
merge ktrace-lwp.
 1.31 15-Oct-2005  yamt - change the way to specify a bufq strategy. (by string rather than by number)
- rather than embedding bufq_state in driver softc,
have a pointer to the former.
- move bufq related functions from kern/subr_disk.c to kern/subr_bufq.c.
- rename method to strategy for consistency.
- move some definitions which don't need to be exposed to the rest of kernel
from sys/bufq.h to sys/bufq_impl.h.
(is it better to move it to kern/ or somewhere?)
- fix some obvious breakage in dev/qbus/ts.c. (not tested)
 1.30 25-Jul-2005  drochner remove unused (hopefully:-) softc fields for disk geometry
 1.29 27-Feb-2005  perry branches: 1.29.4;
nuke trailing whitespace
 1.28 25-Sep-2004  thorpej branches: 1.28.4; 1.28.6;
Add support for wedges to the ATA disk driver.
 1.27 30-Aug-2004  drochner make the software managed bad-sector list optional
(kernel option WD_SOFTBADSECT)
 1.26 14-Dec-2003  thorpej Move the wddone() prototype to wd.c
 1.25 14-Dec-2003  thorpej Garbage-collect __ATA_DISK_PRIVATE; it is no longer needed.
 1.24 14-Dec-2003  thorpej Move struct ata_device from wdvar.h to atavar.h.
 1.23 14-Dec-2003  thorpej Move struct ata_bustype from wdvar.h to atavar.h.
 1.22 14-Dec-2003  thorpej Move struct ata_bio from wdvar.h to atavar.h.
 1.21 07-Nov-2003  mycroft Replace home-grown locks with lockmgr().
 1.20 02-Nov-2003  wiz Fix some typos. From Tom Cosgrove via jmc@openbsd.
 1.19 08-Oct-2003  bouyer Make the ATA mid-layer appears as atabus, as proposed in
http://mail-index.netbsd.org/tech-kern/2003/09/25/0006.html
This adds a device (atabus) between IDE controllers and wd or atapibus, to
have each ATA channel show up in the device tree. Later there will be atabus
devices in /dev, so that we can do IOCTL on them.
Each atabus has its own kernel thread, to handle operations that needs polling,
e.g. reset and others.

Device probing on each bus it defered to the atabus thread creation.
This allows to do the reset and basic device probes in parallel, which reduce
boot time on systems with several pciide controllers.
 1.18 05-Oct-2003  bouyer Remove references to University of California from my copyright notices.
 1.17 15-Apr-2003  darrenr branches: 1.17.2;
Implement (and document!) "dkctl <dev> badsector .." to `manage' the bad
sector information in the kernel. Doing this uncovered some shortcomings
that should have been pretty obvious with the code committed prior, addressing
the major kludge with a new struct - disk_bacsecinfo to be passed into
DIOCBSLIST.
 1.16 15-Apr-2003  darrenr Automatically build up a list of bad sectors from failed read operations and
prempt read operations on matching regions with a failure rather than waiting
for the device to return a failure. The I/O operation must have already failed
MAXRETRIES times before being added to the list - this can generally take up
to 12 seconds.
List is made accessible to userspace via DIOCBSLIST and DIOCBSFLUSH.
 1.15 03-Apr-2003  fvdl Check RAW_PART against the media size instead of the disklabel.
Add the media size in 512-byte sectors to the softc, to avoid
some 64 bit computations. Bump the capacity stored in softcs
for disks to 64 bits.
 1.14 21-Mar-2003  thorpej Work around a buggy PHY in some Seagate S-ATA drives which get confused
by the way some S-ATA controllers packetize data when the sector count of
the transfer mod 15 is 1.
 1.13 27-Jan-2003  thorpej Experimental support for RAID volumes configured by ATA "RAID" controllers.
Such RAID controllers are actually just IDE controllers with a BIOS that
can create RAID volumes and write the configuration info to config blocks
on the disks. The BIOS can do I/O to these volumes, and the OS must
understand the config blocks and implement RAID in software in order to be
able to use these volumes.

Only SPAN (simple concatenation) and RAID0 are supported at this time,
and writing back config blocks is also not supported at this time. Currently,
only the Promise configuration scheme is supported, although supporting
the Highpoint scheme should not be too difficult.

In any case, this is sufficient to use the Promise RAID0 volume (thus
preserving the win2k AS installation) on this new Intel server I have.

Thanks to Soren Schmidt for doing the work in FreeBSD; it made this
task much easier. The config block parsing code is adapted from his
work.
 1.12 23-Apr-2002  bouyer More copyright fixes, pointed out by Thomas. Thanks !
 1.11 13-Jan-2002  christos Add LBA48 support based on patches from Keisuke YOSHIDA. KNF a bit while
I am there.
 1.10 03-Dec-2001  bouyer Update my copyrigth.
 1.9 03-Dec-2001  bouyer Allow wd to attach to something else than wdc/pciide (like USB<->ATA bridges):
- move some functions from ata.c to ata_wdc.c or wdc.c.
- add callbacks to struct ata_bustype so that wd.c doesn't call directly
functions from the lower level driver.
 1.8 02-Dec-2001  bouyer Clean up attach of wd/atapibus:
kill ata_atapi_attach. Change atapibus to use a struct scsipi_channel instead
of ata_atapi_attach as attach arch. Create a ata_device, compatible with
scsipi_channel, to attach wd.
 1.7 08-Jul-2001  wiz branches: 1.7.2;
Correct various misspellings of 'transfer' and inflected forms.
 1.6 15-May-2000  bouyer branches: 1.6.6;
Sync my copyrigth notice
 1.5 01-Apr-2000  bouyer - DMA code cleanup: pciide_dma_finish() doesn't stop/unload the current DMA op
if an IRQ was not detected, unless the force flag was given. Use this to
detect if the IRQ was for us (closer to shared IRQ for controllers which
don't have their own IRQ handler in pciide.c) and to poll for DMA xfer.
Also makes the timeout recovery code simpler.
- ATAPI cleanup: don't call controller-specific functions from atapiconf.c
(wdc_*), so that it's possible to attach an atapibus to something else
than a wdc/pciide (Hi Lennart :).
Overload struct scsi_adapter with struct atapi_adapter, defined
as struct scsi_adapter + atapi-specific callbacks. scsipi_link still points
to an scsi_adapter, atapi code casts it to atapi_adapter if needed.
Move atapi_softc to atapiconf.h so that it can be used by the underlying
controller code (e.g. atapi_wdc.c).
Add an atapi-specific callback *atapi_probedev(), which probe a drive
in a controller-specific way, allocate the sc_link and fills in the
ataparams if needed. It then calls atapi_probedev() (from atapiconf.c)
to do the generic initialisations and attach the device.
- While I'm there merge and centralise the state definitions in atavar.h.
It should now be possible to use a common ata/atapi routine to set the
drive's modes (will do later).
 1.4 20-Oct-1999  enami Cancel active transfers on aic/wdc detach.
Also makes LS-120 drive works for me again.
 1.3 11-Nov-1998  bouyer branches: 1.3.4; 1.3.10; 1.3.12; 1.3.14;
- clearify the boot messages (features supported vs features used). Thanks to
Havard Eidnes for his complains about this :)
- fix some typo in comments
- hoppefully better detection of drives reporting bogus PIO modes.
 1.2 12-Oct-1998  bouyer Merge bouyer-ide
 1.1 04-Jun-1998  bouyer branches: 1.1.2;
file wdvar.h was initially added on branch bouyer-ide.
 1.1.2.2 13-Aug-1998  bouyer - sync with HEAD
- better error detection and report for wd drives
- better wddump() support
- wdcwait() now takes a timout argument, so that we can honnor the
timeout argument of scsipi cmds.
 1.1.2.1 04-Jun-1998  bouyer Commit changes to the IDE system in a branch. This allows a better separation
between higth-level and low-level (i.e. registers read/write) and generalize
the queue for all commands. This also add supports for IDE DMA.
 1.3.14.1 27-Dec-1999  wrstuden Pull up to last week's -current.
 1.3.12.1 15-Nov-1999  fvdl Sync with -current
 1.3.10.2 20-Nov-2000  bouyer Update thorpej_scsipi to -current as of a month ago
A i386 GENERIC kernel compiles without the siop, ahc and bha drivers
(will be updated later). i386 IDE/ATAPI and ncr work, as well as
sparc/esp_sbus. alpha should work as well (untested yet).
siop, ahc and bha will be updated once I've updated the branch to current
-current, as well as machine-dependant code.
 1.3.10.1 20-Oct-1999  thorpej Sync w/ trunk.
 1.3.4.1 07-Jul-2000  he Apply patch (requested by bouyer):
Add support for the following PCIIDE controllers:
o AMD 756
o CMD PCI0648 and PCI0649
o Hightpoint HPT366
o OPTi 82c621 (and a few of its derivatives)
o Promise Ultra/33 and Ultra/66
o Intel 82801 (ICH/ICH0)
Also fix PR#10437 (detect more ATAPI devices).
 1.6.6.4 20-Jun-2002  nathanw Catch up to -current.
 1.6.6.3 28-Feb-2002  nathanw Catch up to -current.
 1.6.6.2 08-Jan-2002  nathanw Catch up to -current.
 1.6.6.1 24-Aug-2001  nathanw Catch up with -current.
 1.7.2.3 23-Jun-2002  jdolecek catch up with -current on kqueue branch
 1.7.2.2 11-Feb-2002  jdolecek Sync w/ -current.
 1.7.2.1 10-Jan-2002  thorpej Sync kqueue branch with -current.
 1.17.2.7 10-Nov-2005  skrll Sync with HEAD. Here we go again...
 1.17.2.6 04-Mar-2005  skrll Sync with HEAD.

Hi Perry!
 1.17.2.5 19-Oct-2004  skrll Sync with HEAD
 1.17.2.4 21-Sep-2004  skrll Fix the sync with head I botched.
 1.17.2.3 18-Sep-2004  skrll Sync with HEAD.
 1.17.2.2 03-Sep-2004  skrll Sync with HEAD
 1.17.2.1 03-Aug-2004  skrll Sync with HEAD
 1.28.6.1 19-Mar-2005  yamt sync with head. xen and whitespace. xen part is not finished.
 1.28.4.1 29-Apr-2005  kent sync with -current
 1.29.4.5 24-Mar-2008  yamt sync with head.
 1.29.4.4 17-Mar-2008  yamt sync with head.
 1.29.4.3 27-Feb-2008  yamt sync with head.
 1.29.4.2 21-Jan-2008  yamt sync with head
 1.29.4.1 21-Jun-2006  yamt sync with head.
 1.32.58.1 11-Dec-2007  yamt sync with head.
 1.32.56.1 26-Dec-2007  ad Sync with head.
 1.32.46.2 23-Mar-2008  matt sync with HEAD
 1.32.46.1 09-Jan-2008  matt sync with HEAD
 1.32.44.2 06-Nov-2007  joerg Refactor PNP API:
- Make suspend/resume directly a device functionality. It consists of
three layers (class logic, device logic, bus logic), all of them being
optional. This replaces D0/D3 transitions.
- device_is_active returns true if the device was not disabled and was
not suspended (even partially), device_is_enabled returns true if the
device was enabled.
- Change pnp_global_transition into pnp_system_suspend and
pnp_system_resume. Before running any suspend/resume handlers, check
that all currently attached devices support power management and bail
out otherwise. The latter is not done for the shutdown/panic case.
- Make the former bus-specific generic network handlers a class handler.
- Make PNP message like volume up/down/toogle PNP events. Each device
can register what events they are interested in and whether the handler
should be global or not.
- Introduce device_active API for devices to mark themselve in use from
either the system or the device. Use this to implement the idle handling
for audio and input devices. This is intended to replace most ad-hoc
watchdogs as well.
- Fix somes situations in which audio resume would lose mixer settings.
- Make USB host controllers better deal with suspend in the light of
shared interrupts.
- Flush filesystem cache on suspend.
- Flush disk caches on suspend. Put ATA disks into standby on suspend as
well.
- Adopt drivers to use the new PNP API.
- Fix a critical bug in the generic cardbus layer that made D0->D3
break.
- Fix ral(4) to set if_stop.
- Convert cbb(4) to the new PNP API.
- Apply the PCI Express SCI fix on resume again.
 1.32.44.1 16-Oct-2007  joerg Replace shutdown hook to flush cache with PNP style power handler to
flush the disk cache also on suspend.
 1.33.10.1 03-Apr-2008  mjf Sync with HEAD.
 1.33.6.1 24-Mar-2008  keiichi sync with head.
 1.36.22.1 21-Apr-2010  matt sync to netbsd-5
 1.36.14.1 10-Jan-2010  snj Pull up following revision(s) (requested by dsl in ticket #1220):
sys/dev/ata/wd.c: revision 1.380
sys/dev/ata/wdvar.h: revision 1.38
ATA sepcs changed the LBA48 boundary from 0xfffffff to 0xffffffe between
ATA6r1 and ATA6r3, which caused drives conform to ATA6r3 or later to
reject LBA28 commands at sector 0xfffffff.
Get the right idea from the LBA48 boundary from IDENTIFY words 60-61.
Remove the WD_QUIRK_FORCE_LBA48 quirk, associated tables entries and
autodetect code, it's not needed any more.
Based on patch sent to teck-kern by Christoph Badura, use of words 60-61
instead of a constant for the LBA48 boundary by me.
 1.36.4.1 11-Mar-2010  yamt sync with head
 1.38.12.1 17-Apr-2012  yamt sync with head
 1.39.2.1 18-Feb-2012  mrg merge to -current.
 1.40.6.1 03-Dec-2017  jdolecek update from HEAD
 1.41.2.2 05-Oct-2016  skrll Sync with HEAD
 1.41.2.1 06-Jun-2015  skrll Sync with HEAD
 1.42.2.1 26-Jul-2016  pgoyette Sync with HEAD
 1.43.4.10 07-Oct-2017  jdolecek make usage of NCQ 'high' priority for BPRIO_TIMECRITICAL xfers settable via
sysctl, too
 1.43.4.9 28-Sep-2017  jdolecek add sysctls to control if NCQ is being used, and how many max tags;
I have a drive which is significantly slower with NCQ than non-NCQ,
and it's generally useful to have this easily overridable

while here, also move the frequency settings for WD_CHAOS_MONKEY
to a sysctl and make it per-drive
 1.43.4.8 02-Sep-2017  jdolecek include opt_wd.h instead of opt_wd_softbadsect.h, the option doesn't have private file any more
 1.43.4.7 19-Jul-2017  jdolecek tighen and expand error handling, mostly for NCQ use cases:
- make retry timeout callout per xfer, i.e. retry separately
- zero whole bio struct on retry to avoid more stale state
- add a REQUEUE option, which doesn't bump retry count
- add ata_read_log_ext_ncq() for NCQ recovery
- adjust logic for activating xfers - allow next command only when
it's for same drive, several concurrent are only supported when HBA
and driver support FIS-based switching
- add new ata_timeout() which handles race between callout_stop()
and the invokation, add appropriate handling on deactivate/free paths
- stop using ch_status/ch_error in non-wdc code; later it will be dropped
completely
 1.43.4.6 23-Jun-2017  jdolecek restart I/O processing after freeing xfer, i.e. now even after commands
like cache flush or standby; the command handling no longer use on-stack xfer,
hence use queue slot and compete with normal I/O for the xfers

the restart give change to all drives attached to the same channel
in round-robin fashion, for fair usage and to recover from situation
when disk is idle due to all xfers being consumed by other drives

make special concession for flush cache - ignore any new I/O requests
on the particular disk while the flush cache is waiting for xfer,
so that I/O queue won't starve the flush cache and the flush cache would
be done ASAP

tested on piixide(4), ahci(4), siisata(4)
 1.43.4.5 23-Apr-2017  jakllsch Simplify wi_find() with container_of().

No need to search a list of wd_ioctls when the *bp is always going to be within the wd_ioctl structure.

Correctly initialize wi->wi_bp.b_dev in the non-data case anyway.

Fixes ATAIOCCOMMAND procesing on driver instances other than wd0.
 1.43.4.4 19-Apr-2017  jdolecek adjust ata code to support more than one active command, including the
timeout handling, add support for NCQ commands

move probe for NCQ and number of tags to middle layer, negotiate mutual
support between drive and controller

implement NCQ support in ahci(4)
 1.43.4.3 15-Apr-2017  jdolecek probe for NCQ and WRITE DMA FUA EXT support
 1.43.4.2 15-Apr-2017  jdolecek make ata_xfer's allocated as part of ata_queue and make it include ata_bio;
they are pre-allocated on attach and ata_get_xfer() now never sleep, drop the
pool

modify wd(4) to file the bio requests using the xfers and hence
make it possible to have more than one active I/O request in flight;
ata_bio callback doesn't need to allocate any memory any more,
require it to never return ATACMD_TRY_AGAIN

move lp, badsect, multi from ata_bio to ata_drive_datas, as they are per-drive,
not per transfer

drop unused drv atac_claim_hw/atac_free_hw hooks, and also drop again ata_bio
c_hwslot
 1.43.4.1 12-Apr-2017  jdolecek convert to mutex, mark MPSAFE
 1.46.6.4 22-Sep-2018  jdolecek remove explicit ata_channel_start() calls, it's no longer necessary
now that ata_xfer's are allocated via pool and not really limited;
replace by just a callout to restart the processing for rare cases
where system runs out of memory
 1.46.6.3 22-Sep-2018  jdolecek fix use-after-free in wd(4) dump, detected by switch to the pool

change code in wd_dumpblocks() to use it's own non-pool ata_xfer,
which skips the deallocation step and thus keeps the contents when the I/O
is finished
 1.46.6.2 22-Sep-2018  jdolecek separate ata_xfer slot allocation and the memory allocation, so that
there can be more queued xfers than number of supported slots by controller,
and use a pool instead of custom pre-allocation

primarily to help PR kern/52614

remove no longer needed custom wd(4) logic for flush cache

switch also wd(4) trim/suspend/setcache/wdioctlstrategy to sleep waiting
for the memory, they are all called from process context and this
avoids spurious failures
 1.46.6.1 31-Aug-2018  jdolecek refactor ata_xfer to be just dumb structure; move all callouts/condvars out

retry callout to wd(4); reset callout and the active/cmd finish condvars
to channel queue; change code using the condvars so it works if there
are multiple waiters

simplify the async wait code for cmds, replace ata_wait_xfer()/ata_wake_xfer()
with ata_wait_cmd()

fix the callout_invoking/ack race handling code for timeouts to
actually have chance to work; change mvsata(4) to use generic timeout func

towards resolution of kern/52614
 1.46.4.2 08-Apr-2020  martin Merge changes from current as of 20200406
 1.46.4.1 10-Jun-2019  christos Sync with HEAD
 1.46.2.1 26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.49.4.1 21-Mar-2020  martin Pull up following revision(s) (requested by riastradh in ticket #788):

sys/sys/dkio.h: revision 1.26
sys/dev/dkwedge/dk.c: revision 1.100
sys/sys/disk.h: revision 1.75
external/cddl/osnet/dist/uts/common/fs/zfs/vdev_disk.c: revision 1.14
external/cddl/osnet/dist/uts/common/fs/zfs/vdev_disk.c: revision 1.15
sys/dev/cgd.c: revision 1.121
sys/dev/ata/wdvar.h: revision 1.50
sys/kern/subr_disk_open.c: revision 1.15
sys/dev/ata/wd.c: revision 1.459

New ioctl DIOCGSECTORALIGN returns sector alignment parameters.

struct disk_sectoralign {
/* First aligned sector number. */
uint32_t dsa_firstaligned;
/* Number of sectors per aligned unit. */
uint32_t dsa_alignment;
};

- Teach wd(4) to get it from ATA.
- Teach cgd(4) to pass it through from the underlying disk.
- Teach dk(4) to pass it through with adjustments.
- Teach zpool (zfs) to take advantage of it.
=> XXX zpool doesn't seem to understand when the vdev's starting
sector is misaligned.

Missing:
- ccd(4) and raidframe(4) support -- these should support _using_
DIOCGSECTORALIGN to decide where to start putting ccd or raid
stripes on disk, and these should perhaps _implement_
DIOCGSECTORALIGN by reporting the stripe/interleave factor.
- sd(4) support -- I don't know any obvious way to get it from SCSI,
but if any SCSI wizards know better than I, please feel free to
teach sd(4) about it!
- any ld(4) attachments -- might be worth teaching the ld drivers for
nvme and various raid controllers to get the aligned sector size

There's some duplicate logic here for now. I'm doing it this way,
rather than gathering the logic into a new disklabel_sectoralign
function or something, so that this change is limited to adding a new
ioctl, without any new kernel symbols, in order to make it easy to
pull up to netbsd-9 without worrying about the module ABI.

Make getdiskinfo() compatible with a DIOCGWEDGEINFO.

dkw_parent is defined to hold the disk name as used by disk_find(), not
a partition (i.e. no partition letter appended).

Use utility functions to handle disk geometry.
 1.51.10.1 02-Aug-2025  perseant Sync with HEAD

RSS XML Feed