Home | History | Annotate | Download | only in ic
History log of /src/sys/dev/ic/ahcisata_core.c
RevisionDateAuthorComments
 1.108  10-Sep-2023  abs Rework AHCISATA_EXTRA_DELAY for kern/56737

- Remove AHCI_QUIRK_EXTRA_DELAY as issue appears to be drive and
not controller related
- Replace AHCISATA_EXTRA_DELAY with AHCISATA_REMOVE_EXTRA_DELAY,
so defaulting to enabling the extra delay, as the downside of
slower probing on systems which do not need it is less than having
other systems intermittently fail to probe and attach drives
- Also allow disabling extra delay with AHCISATA_EXTRA_DELAY_MS = 0

We should return to this code to work out which of the extra delays
are needed, and how long they need to be. It may be that faster
systems are more likely to trigger the issue (I've only seen it on
a 13th gen i7-13700, though only tested on a limited set)

XXX pullup -10
 1.107  01-Aug-2022  mlelstv branches: 1.107.4;
Revert last accidental commits.
 1.106  01-Aug-2022  mlelstv Also fix shift values for SCT constants.
 1.105  19-Nov-2021  rin ahcisata(4): Introduce AHCI_QUIRK_EXTRA_DELAY quirk for devices that
need extra delays as done by AHCISATA_EXTRA_DELAY option.

Enable this quirk for "C600/X79 AHCI". Also add commented out quirk
entries for "Bay Trail SATA (AHCI)" and "Mobile AHCI SATA Controller",
for which non-reproducible failures worked around by extra delays have
been reported.

500 ms of delays inserted by these option/quirk may be too much. Add
AHCISATA_EXTRA_DELAY_MS option to adjust number of delays in ms, like:

----
options AHCISATA_EXTRA_DELAY_MS=200
----

Thanks prlw1@ and jun@ for testing!
 1.104  10-Nov-2021  msaitoh s/endianess/endianness/
 1.103  11-Oct-2021  jmcneill ahcisata: remove excessive delays from drive probe path

There are a handful of inexplicable 500ms delays introduced to the drive
detect path in this driver, slowing boot. They can be re-enabled with
options AHCISATA_EXTRA_DELAY, but should not be enabled for normal kernels.
If a delay does need to be introduced in these places, the value should
either be more carefully selected or the scope limited to hardware that
requires the extra delay.
 1.102  05-Oct-2021  rin PR kern/56403

Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.

(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.

Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().

(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).

"Go ahead" by jdolecek@.
 1.101  03-Sep-2021  mrg revert "make ahcisata(4) work on rk3399 (rockpro64)" from 23-06-2021.

this problem is known to affect more than AHCI, and a more general
solution has been commited now.
 1.100  07-Aug-2021  thorpej Merge thorpej-cfargs2.
 1.99  23-Jun-2021  mrg branches: 1.99.2;
make ahcisata(4) work on rk3399 (rockpro64)

on rk3399, a marvell 9230 ahci sata card consistently takes between
1213 and 1216 milliseconds, the ahci spec says this should complete
in 1000 or fewer.

add a "pcie-reset-ms" uint32 property that ahcisata defaults to 1000
if not set, and the rockchip platform code sets to 2000.


ok @jmcneill
 1.98  24-Apr-2021  thorpej branches: 1.98.2;
Merge thorpej-cfargs branch:

Simplify and make extensible the config_search() / config_found() /
config_attach() interfaces: rather than having different variants for
which arguments you want pass along, just have a single call that
takes a variadic list of tag-value arguments.

Adjust all call sites:
- Simplify wherever possible; don't pass along arguments that aren't
actually needed.
- Don't be explicit about what interface attribute is attaching if
the device only has one. (More simplification.)
- Add a config_probe() function to be used in indirect configuiration
situations, making is visibly easier to see when indirect config is
in play, and allowing for future change in semantics. (As of now,
this is just a wrapper around config_match(), but that is an
implementation detail.)

Remove unnecessary or redundant interface attributes where they're not
needed.

There are currently 5 "cfargs" defined:
- CFARG_SUBMATCH (submatch function for direct config)
- CFARG_SEARCH (search function for indirect config)
- CFARG_IATTR (interface attribte)
- CFARG_LOCATORS (locators array)
- CFARG_DEVHANDLE (devhandle_t - wraps OFW, ACPI, etc. handles)

...and a sentinel value CFARG_EOL.

Add some extra sanity checking to ensure that interface attributes
aren't ambiguous.

Use CFARG_DEVHANDLE in MI FDT, OFW, and ACPI code, and macppc and shark
ports to associate those device handles with device_t instance. This
will trickle trough to more places over time (need back-end for pre-OFW
Sun OBP; any others?).
 1.97  04-Mar-2021  skrll branches: 1.97.2;
Use BUS_ADDR_{LO,HI}32. NFC.
 1.96  04-Mar-2021  skrll Use __BIT(). NFC.
 1.95  11-Jan-2021  skrll KNF
 1.94  29-Dec-2020  skrll Some more whitespace consistency / KNF
 1.93  29-Dec-2020  skrll Function declaration formating whitespace consistency. NFCI.
 1.92  28-Dec-2020  jmcneill Remove the AHCI_QUIRK_SKIP_RESET quirk now that the underlying issue is
fixed.
 1.91  28-Dec-2020  jmcneill Make sure to ack IS after PxIS when polling and when using multiple MSI-X
messages.
 1.90  27-Dec-2020  jmcneill AHCI 1.3.1 section 5.5.3 "Processing Completed Commands" says that we
should clear PxIS before IS.IPS.
 1.89  26-Dec-2020  jmcneill ahci_intr: use ffs in the port bitmask instead of looping over all 32 bits
 1.88  26-Dec-2020  jmcneill AHCI 1.3.1 specification says that it is good practice for system software
to 'zero-out' the memory allocated and referenced by PxCLB and PxFB.
 1.87  25-Dec-2020  skrll Trailing whitespace
 1.86  25-Dec-2020  skrll Use designated initializers for struct ata_bustype
 1.85  20-Dec-2020  jmcneill Retry clearing WDCTL_RST a few times before giving up. Makes SATA work in
Solidrun Honeycomb LX2K.
 1.84  19-Dec-2020  jmcneill ahci_exec_fis: wait for the correct amount of time when AT_WAIT is set
 1.83  13-Apr-2020  jdolecek branches: 1.83.2;
fix use-after-free for ata xfer on bio submission found by KASAN

driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself

PR kern/55169 by Nick Hudson
 1.82  19-Feb-2020  riastradh branches: 1.82.4;
C99 initializers for scsipi_bustype. No functional change intended.
 1.81  18-Jan-2020  simonb Add new AHCI_QUIRK_BADNCQ quick for controllers that have issues with
NCQ on (some) drives. Enable this quirk for ATI (AMD) SB600/SB700
controllers. Alternate fix for kern/54790 and kern/54855.

ok jdolecek@, tested on my SB700 chipset and tsutsui's SB600 chipset.
 1.80  27-Dec-2019  msaitoh branches: 1.80.2;
s/transfered/transferred/
 1.79  22-Dec-2019  jmcneill When resetting a drive, if the command list is running and CLO is not
supported, attempt to stop the drive first and fail gracefully if that
fails instead of triggering a KASSERT on DIAGNOSTIC kernels.
 1.78  29-Sep-2019  jakllsch Use AHCI_RFIS_SYNC in ahci_cmd_complete() AT_READREG case.

Ensures CPU sees the current device to host FIS before parsing.
 1.77  29-Sep-2019  jakllsch Use AHCI_CMDTBL_SYNC in ahci_exec_fis() to ensure hardware sees the
command FIS.
 1.76  29-Sep-2019  jakllsch Ensure cmdh_prdtl is 0 in ahci_do_reset_drive().

(I'm probably just being paranoid here.)
 1.75  07-Apr-2019  bouyer branches: 1.75.4;
If Command List Override is not supported by the controller, the FIS setting
the WDC_RST bit will fail if the drive already has BSY or DRQ set.
In this case, reset the channel and retry the drive reset.
With this, the drive reset will succeed after a atactl sleep command on
a controller without the CLO feature.
 1.74  18-Jan-2019  jdolecek revert rev 1.73 - while it apparently helped Christos case, it re-broke
the disk detection on several other systems, so need something else
 1.73  12-Jan-2019  jdolecek partially back-off rev. 1.68 - when the drive reset for port 15 fails,
on some systems controller isn't actually able to process further commands,
it's really necessary to do explicitely reset for drive 0 too

towards resolution of PR kern/53307
 1.72  07-Dec-2018  jdolecek add optional hook for intr establish when active port is attached, export
ahci_intr_port() in form suitable for interrupt hanlder, and probe for GHC
MRSM flag as courtesy for use by the intr hook

towards multi-vector MSI/MSI-X support
 1.71  20-Nov-2018  jdolecek s/failing/failed/ for the aprint_error()
 1.70  20-Nov-2018  jdolecek also set WDCTL_4BIT for the softreset, seems this one is actually also
required to fully resolve the problem

PR kern/48214 PR kern/53307 PR kern/53524
 1.69  19-Nov-2018  jdolecek during SRST (softreset), make a short delay between the RST set and
clear; SATA specifies minimum 5 usec for the toggle period, and
some controllers (seems usually on AMD motherboards) actually require it

this fixes the 'clearing WDCTL_RST failed' error in PR kern/53307
and PR kern/53524

confirmed working on a ASUS Prime A320M-K mainboard by me, and by Patrick
Welche on another Ryzen system

XXX pullup-8
 1.68  19-Nov-2018  jdolecek if softreset during PMP detection fails, disable PMP, reset port and continue
with sig detected after the initial COMRESET

this does not yet fix the infamous 'clearing WDCTL_RST failed', but at least
now the disk is detected and usable when it happens

tested on AMD system with ASUS Prime A320M-K, similar to one from PR kern/53524
 1.67  19-Nov-2018  jdolecek consistently use the SATA correct 'port %d' instead of 'channel', some
whitespace fixes
 1.66  02-Nov-2018  jdolecek fix ahci_detach() to count the ports propertly (same as attach), to avoid
triggerring panic when disks are plugged to non-adjacent ports - this should
fix panic reported by Masanobu SAITOH on current-users

also fix different miscounting of ports in ahci_setup_ports() and
ahci_reprobe_drives()
 1.65  24-Oct-2018  jdolecek detach the controller itself on shutdown; adjust to not detach already
detached atabus/channel
 1.64  22-Oct-2018  jdolecek Merge jdolecek-ncqfixes branch

- ata_xfer's are dynamicall allocated as needed using a pool, no longer
limited to number of possible openings supported by controller; dump
and recovery paths use dedicated pre-allocated storage
- moved callouts and condvars from ata_xfer to queue or channel, so that
ata_xfer does not need special initialization
- slot allocation now done when xfer is being activated, uncoupled
from memory allocation; active slots are no longer tracked by controller
code
- channel and drive reset is done always via the atabus thread, and
now executes with channel locked the whole time
- NCQ recovery moved to shared function, and run via the thread also
- added some workarounds for buggy error recovery AHCI emulation in QEMU
and Parallels

designed to primarily fix kern/52614, but might also help with kern/47041
and kern/53183
 1.63  18-Sep-2018  jdolecek fix ata_xfer leak on drive probe
 1.62  09-Jul-2018  kamil branches: 1.62.2;
Avoid undefined behavior of signedness bit shift in ahcisata_core.c

sys/dev/ic/ahcisata_core.c:365:31, left shift of 1 by 31 places cannot be represented in type 'int'
sys/dev/ic/ahcisata_core.c:558:16, left shift of 1 by 31 places cannot be represented in type 'int'

Detected with Kernel Undefined Behavior Sanitizer.

This code could be refactored in future and switched to ISSET(9) API,
instead of reinventing the common functionality.
 1.61  08-Jul-2018  jdolecek whitespace fix
 1.60  11-Nov-2017  jdolecek branches: 1.60.2; 1.60.4;
change several of error logs in ahci_intr_port() to use AHCIDEBUG_PRINT()
so they don't show unless debugging; e.g. the AHCI_P_IX_TFES seems to be
triggered normally by ATAPI with 'Media Change' error

adresses 'dmesg spam: ahcisata0 port 1: active 2 is 0x40000001 tfd 0x2051'
thread by Stefan Hertenberger on current-users@
 1.59  20-Oct-2017  jdolecek move ata_queue_alloc(1) and ata_queue_free() calls to ata_channel_init()
and ata_channel_destroy() respectively, to make attachment code simpler,
and to make it easier to spot special queue manipulation like cmdide(4)

on topic of PR kern/52606
 1.58  07-Oct-2017  jdolecek Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.57  03-Jun-2016  jmcneill branches: 1.57.6;
PR kern/51211: atactl atabus0 reset causes a panic on Tegra K1

Fix an issue where ahci_reset_channel calls ahci_channel_start with clo=1
even if CAP.SCLO=0.
 1.56  02-May-2016  christos move scsipi_strvis -> libkern:strnvisx()
change the prototype to match userland
fix sizes of strings passed to it
 1.55  11-Nov-2015  jakllsch Only sync and unload dma map in ahci_atapi_complete() if appropriate.

Fixes ahcisata atapi cd(4) on arm (specifically, JETSONTK1), which insists
that a zero-length dmamap can not be synced.
 1.54  24-May-2015  jmcneill Add AHCI_QUIRK_SKIP_RESET quirk to allow for skipping the drive reset
sequence in ahci_do_reset_drive.
 1.53  04-Dec-2014  joerg Increase reset delay in polling mode to 10ms per loop. It seems like
AHCI on the Cubietruck doesn't like the tigther loop.

PR 49448.
 1.52  23-Nov-2014  joerg branches: 1.52.2;
Provide the timeout argument to ahci_exec_fis in ms. Use 1ms delays
when we are not allowed to sleep, full ticks otherwise.

Bump the timeout for reset to 100ms, 10ms is not enough on the
Cubietruck.
 1.51  24-Feb-2014  jmcneill - Only enable AHCI mode if it is not already enabled.
- Add support for capturing initial CAP/CAP2/PI regs before reset (and
restoring them afterwords)
- Add optional callbacks for channel_start / channel_stop.
 1.50  08-Sep-2013  matt Allow the attachment to override ahci_ports.
Keep ACHI_PI cached in the softc.
 1.49  08-Sep-2013  matt If there is only 1 port, don't say "1 ports".
Add a debug printf after reading the active ports.
 1.48  22-Jun-2013  matt branches: 1.48.2;
Make sure dynamically allocated channel_queue struct are zero-filled.
 1.47  03-Apr-2013  bouyer Fix kernel dump on ahci controller, by making sure we won't sleep
while dumping:
- introduce ata_delay() which either use delay() or kpause()
depending on flags. use it in sata_reset_interface() and
some ahci functions
- kill ATA_NOSLEEP, it was tested but never set. use ATA_POLL instead.
- reduce delay while polling in ahci, to speed up the dump

Should fix PR kern/41095
 1.46  02-Feb-2013  matt Use __SHIFTOUT to simplify an expression.
 1.45  26-Oct-2012  bouyer Workaround PR kern/47097: use delay() instead of tsleep() ahci_do_reset_drive()
if not called with AT_WAIT.
The right fix here is to change the ata layer to reset the drive from
thread context, to avoid a 0.5 delay() in interrupt context when a drive
fails.
 1.44  27-Sep-2012  matt Add some more debugging messages.
 1.43  20-Aug-2012  bouyer branches: 1.43.2;
Fix typo, pointed out by Markus W Kilbinger
 1.42  20-Aug-2012  bouyer Fix AHCI_QUIRK_BADPMPRESET: when reset fails for drive 15, stop, clear
errors and restart the channel before retrying with drive 0.
Should fix the problem of drives not detected any more reported by
Markus W Kilbinger on current-users.
 1.41  10-Aug-2012  bouyer Work around some SATA PMP issues in some AHCI controllers by either
disabling PMP entirely, or special handling in the reset function.
Controller list from linux and FreeBSD.
 1.40  31-Jul-2012  bouyer Apply back changes that were reverted on Jul 24 and Jul 26 (general ata/wdc
cleanup and SATA PMP support), now that I'm back to fix the fallouts.
 1.39  26-Jul-2012  jakllsch Revert, with intention of restoring in a less invasive way, the SATA Port
Multiplier code.

ok christos@
 1.38  24-Jul-2012  jakllsch Revert dsl@'s changes of Sun, 15 Jul 2012 10:55:35 +0000 and
Sun, 15 Jul 2012 10:56:50 +0000, excepting the kernel version bump.
First step in reverting regressions to ata(4) subsystem during the addition of
port multiplier support.
 1.37  15-Jul-2012  dsl Some namespace protection (and add greppablity).
Prefix the DRIVE_ and DRIVET_ constants from atavar.h with ATA_.
Don't use an enum for drive_type - you don't know how big it will be.
Move driver_type to avoid implicit structure padding (esp on arm).
This change is purely lexical and mechanical.

Update to 6.99.9 - this wasn't done when the SATA PMP changes
were made - I'm sure they warranted a bump.
 1.36  09-Jul-2012  dsl This a 'quick fix' for the breakage in rev 1.35.
Patch from Manuel on current-users mailing list.
Probably requires further investigation, but is a little nicer than
reverting the previous change.
 1.35  02-Jul-2012  bouyer Add sata Port MultiPlier (PMP) support to the ata bus layer,
as described in
http://mail-index.netbsd.org/tech-kern/2012/06/23/msg013442.html
PMP support in integrated to the atabus layer.
struct ata_channel's ch_drive[] is not dynamically allocated, and ch_ndrive
(renamed to ch_ndrives) closely reflects the size of the ch_drive[] array.
Add helper functions atabus_alloc_drives() and atabus_free_drives()
to manage ch_drive[]/ch_ndrives.
Add wdc_maxdrives to struct wdc_softc so that bus front-end can specify
how much drive they really support (master/slave or single).
ata_reset_drive() callback gains a uint32_t *sigp argument which,
when not NULL, will contain the signature of the device being reset.
While there, some cosmetic changes:
- added a drive_type enum to ata_drive_datas, and stop encoding the
probed drive type in drive_flags (we were out of drive flags anyway).
- rename DRIVE_ATAPIST to DRIVE_ATAPIDSCW to better reflect what this
really is
- remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas
- factor out the interpretation of SATA signatures in sata_interpet_sig()

propagate these changes to the ATA HBA drivers, and add support for PMP
to ahcisata(4) and siisata(4).

Thanks to:
- Protocase (http://www.protocase.com/) which provided a system
with lots of controllers, SATA PMP and drive slots
- Conservation Genomics Laboratory, Department of Biology, New Mexico State
University for hosting the above system
- Brook Milligan, who set up remote access and has been very responsive
when SATA cable move was needed
 1.34  20-Apr-2012  bouyer Add a bustype_async_event_xfer_mode() callback to scsipi_bustype (which can
be NULL), so that transport-specific details of transfer mode setting/printing
can be handled more easily.
Move scsipi_async_event_xfer_mode() and scsipi_print_xfer_mode() to
scsi_base.c and split in parallel scsi and FC/SAS parts.
size of struct scsipi_bustype has changed, welcome to 6.99.5
 1.33  10-Jan-2012  jakllsch branches: 1.33.4;
When a port reset fails, it is important to know which port.
 1.32  20-Aug-2011  jakllsch branches: 1.32.2; 1.32.6;
Rework version printout to be more future proof.
Rework features printout using snprintb.
 1.31  10-Jan-2011  tsutsui Don't call bus_dmamap_load(9) and bus_dmamap_sync(9) on command xfers
if (AT_READ|AT_WRITE) in ata_c->flags is set but ata_c->bcount is zero.
Someone actually tries to put such a command and it causes
DIAGNOSTIC panic in x86/bus_dma.c:_bus_dmamap_sync().
I think bus_dma(9) API itself may allow calls with mapsize==0
but there are many MD code that asserts offset>=mapsize or len==0.

The problem is reported and fix is confirmed by Takuro KUBOTA
with XEN DOM0 kernel (which has options DIAGNOSTIC).
 1.30  13-Nov-2010  uebayasi Don't pull in the whole uvm(9) API to access only PAGE_SIZE and
some other constants. These are provided by sys/param.h now.
 1.29  27-Jul-2010  jakllsch Support detachment of ahcisata(4).
Use use 64-bit DMA tag (where available) for ahcisata(4) at jmide(4).
Beginnings of detach/resume support for jmide(4).
Sprinkle static. Misc. little changes.
 1.28  20-Jul-2010  jakllsch Prepare ahcisata(4) to work with 64-bit bus_dma_tag_t.
 1.27  20-Jul-2010  jakllsch Convert ahcisata(4) to C99 exact-width integer types.
 1.26  07-Apr-2010  jakllsch satafis:
- Add function to parse RDH FIS for use in implementing AT_READREG.
- Correct and clean up some structure definitions.
- Sprinkle a bit of const.
- Remove dependency on <dev/ic/wdcreg.h>, WDCTL_4BIT doesn't seem
to be specified by any recent ATA standard, and it seems to make
no difference in practice.
- Stop using WDSD_IBM, these bits have been obsolete since before SATA.

siisata:
- Implement AT_READREG.
- Reap now-unneeded <dev/ic/wdcreg.h>.
- Whitespace and slight debug code cleanup.
- Some possibly-uncessary code reordering.

ahcisata:
- Implement AT_READREG.
- Reap now-unneeded <dev/ic/wdcreg.h>.
 1.25  12-Mar-2010  jakllsch branches: 1.25.2;
ata(4) expects IDENTIFY data to be in host endianess.
Logic borrowed from siisata(4) (which I've confirmed works on sparc64).
Should fix PR kern/39659.
 1.24  10-Mar-2010  bouyer ahcisata_core.c 1.23 cause long drive probe (31s, reported by
Roy Marples), or other devices misbehavior probably due to interrupts issues
(reported by Jukka Ruohonen). Back it out and do the following changes:
- clear port interrupt register before ahci_channel_start() which enables
interrupts
- wait 500ms after sata_reset_interface() before touching SERROR register.
This is what seems to fix the issue I'm seeming on ESB2 controller.
- The 31s delay didn't cause the probe to fail because of a mismatch
in loop index comparison; use a #define for delay after reset
instead of numeric values, to avoid this kind of bugs in the
future.
 1.23  23-Feb-2010  bouyer Move ahci_channel_start() after BSY has cleared (what seems to matter
here is not touching SError while BSY is asserted). Fix random
behavior (serial console corruption/misfunction, misbehavior of USB
controllers, crasy interrupts crashing the Xen hypervisor, ...)
of Intel ESB2 controller with some brand of hitachi drives.
 1.22  21-Nov-2009  jakllsch branches: 1.22.2;
Change bus_space_subregion() size for each SATA PHY register from 1 to 4,
as we do a bus_space_read_4() within these subregions.
 1.21  19-Oct-2009  bouyer Remove closes 3 & 4 from my licence. Lots of thanks to Soren Jacobsen
for the booring work !
 1.20  17-Jun-2009  jakllsch Use abstracted FIS code for ahcisata(4). Have the FIS code pre-zero the FIS.
 1.19  12-Feb-2009  bouyer When an error is reported on a write, data may have been transfered
to the device's cache anyway and so cmdh_prdbc reports a completed
transfer. If we use it to update ata_bio->bcount this has 2 conseqences:
- the automatic LBA48 workaround doesn't qick in because bcount is used
to compute the last sector of the transfer (wd(4) part of kern/40569)
- wd(4) will report a B_ERROR buffer with a b_resid of 0, which panics
a DIAGNOSTIC kernel
Fix by ignoring cmdh_prdbc if we had a write with errors, and in this case
leave ata_bio->bcount at its initial value.

While there use NOERROR instead of 0 for ata_bio->error (cosmetic).

thanks to Matthias Scheler for tests.
 1.18  03-Oct-2008  bouyer branches: 1.18.2; 1.18.4; 1.18.8;
Make sure the adapter is ready to accept FISs before bringing the PHY up.
Wait for the BSY bit to clear in the status register.
Makes the cdrom drive probe properly on a ATI SB600 controller in AHCI
mode.
 1.17  01-Aug-2008  dillo Increase delay after channel reset from 100ms to 500ms, suggested by bouyer.
Fixes unreliable drive detection on NVIDIA MCP67.
 1.16  07-Jun-2008  bouyer branches: 1.16.4;
For ATAPI, in addition to storing the short sense, report a SCSI_CHECK
condition, to that the scsipi layer will issue a request sense.
 1.15  07-May-2008  bouyer branches: 1.15.2;
Wait 100ms after we resseting the PHY and found that a device is preent.
Some device seems to need a bit of delay before being ready to accept a
command (such as my SATA cdrom drive) May fix port-i386/38548.
As a bonus this makes the dmesg output prettier, by not mixing PHY
status with ATA/ATAPI probe.
 1.14  24-Mar-2008  cube branches: 1.14.2; 1.14.4;
Split device_t and softc for atapibus(4).
 1.13  18-Mar-2008  cube Split device_t and softc for ATA devices, as well as wd(4). Other
cosmetic changes where appropriate.
 1.12  11-Feb-2008  xtraeme branches: 1.12.2; 1.12.6;
Attach ahcisata(4) to any RAID controller that supports AHCI, and make it
pass the ATAC_CAP_RAID capability for them. ok'ed by bouyer@.
 1.11  25-Jan-2008  xtraeme Make it print the correct version for AHCI 1.2 controllers.
 1.10  04-Dec-2007  spz try fixing up broken revision string again
 1.9  04-Dec-2007  spz fix up broken revision string
 1.8  12-Nov-2007  bouyer Add ATAPI support to ahcisata(4). lightly tested with a DVD burner, by
mounting a CD and reading from it, and blanking a DVD-RW using cdrecord.
 1.7  12-Nov-2007  joerg Merge refactoring from jmcneill-pm.
 1.6  11-Nov-2007  bouyer Move a delay() inside a loop, so that the 1s delay is really 1s.
 1.5  16-Sep-2007  bouyer branches: 1.5.4; 1.5.6;
Fix various error handling bugs:
- the value of the ATA error register would be computed wrongly, leading to
bogus error values reported to wd(4)
- the channel would not always be restarted after an error, so the next
command would not be handled by the controller
- a timeout condition would not be properly reported to wd(4), leading
to a short transfer instead of a reset/retry
these bugs would cause a AHCI SATA channel to be stalled (no more command
processed) after a "ID not found" or "Aborted command" error reported by the
drive.
 1.4  09-Jul-2007  ad branches: 1.4.2; 1.4.8; 1.4.10; 1.4.12; 1.4.14; 1.4.16;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements
 1.3  25-Jun-2007  bouyer We're not ready to handle ATAPI yet so just claim there's no drive to the
upper layer. This should work around a NULL pointer dereference when
an ATAPI device is detected on a AHCI device.
 1.2  21-Jun-2007  fvdl * Add bus_dmamap_sync calls for cmd structure.
* During initialization, use the right port index when setting up the
physical pointers for a port. Fixes issue with non-contigous ports.

Reviewed by Manuel.
 1.1  12-May-2007  bouyer branches: 1.1.2; 1.1.4;
Split the ahcisata driver in pci front-end and bus-independant back-end.
 1.1.4.5  09-Oct-2007  ad Sync with head.
 1.1.4.4  15-Jul-2007  ad Sync with head.
 1.1.4.3  09-Jun-2007  ad Sync with head.
 1.1.4.2  09-Jun-2007  ad Sync with head.
 1.1.4.1  12-May-2007  ad file ahcisata_core.c was added on branch vmlocking on 2007-06-09 21:37:13 +0000
 1.1.2.2  17-May-2007  yamt sync with head.
 1.1.2.1  12-May-2007  yamt file ahcisata_core.c was added on branch yamt-idlelwp on 2007-05-17 13:41:24 +0000
 1.4.16.7  24-Mar-2008  yamt sync with head.
 1.4.16.6  11-Feb-2008  yamt sync with head.
 1.4.16.5  04-Feb-2008  yamt sync with head.
 1.4.16.4  15-Nov-2007  yamt sync with head.
 1.4.16.3  27-Oct-2007  yamt sync with head.
 1.4.16.2  03-Sep-2007  yamt sync with head.
 1.4.16.1  09-Jul-2007  yamt file ahcisata_core.c was added on branch yamt-lazymbuf on 2007-09-03 14:34:13 +0000
 1.4.14.4  03-Jun-2008  skrll Sync with netbsd-4.
 1.4.14.3  23-Sep-2007  wrstuden Sync with somewhat-recent netbsd-4.
 1.4.14.2  03-Sep-2007  wrstuden Sync w/ NetBSD-4-RC_1
 1.4.14.1  09-Jul-2007  wrstuden file ahcisata_core.c was added on branch wrstuden-fixsa on 2007-09-03 07:04:22 +0000
 1.4.12.7  26-Feb-2009  snj Pull up following revision(s) (requested by bouyer in ticket #1277):
sys/dev/ic/ahcisata_core.c: revision 1.19
When an error is reported on a write, data may have been transfered
to the device's cache anyway and so cmdh_prdbc reports a completed
transfer. If we use it to update ata_bio->bcount this has 2 conseqences:
- the automatic LBA48 workaround doesn't qick in because bcount is used
to compute the last sector of the transfer (wd(4) part of kern/40569)
- wd(4) will report a B_ERROR buffer with a b_resid of 0, which panics
a DIAGNOSTIC kernel
Fix by ignoring cmdh_prdbc if we had a write with errors, and in this case
leave ata_bio->bcount at its initial value.
While there use NOERROR instead of 0 for ata_bio->error (cosmetic).
thanks to Matthias Scheler for tests.
 1.4.12.6  23-Mar-2008  jdc Pull up revision 1.8 (requested by bouyer in ticket #1085).

Add ATAPI support to ahcisata(4). lightly tested with a DVD burner, by
mounting a CD and reading from it, and blanking a DVD-RW using cdrecord.
 1.4.12.5  23-Mar-2008  jdc Pull up revision 1.6 (requested by bouyer in ticket #1082).

Move a delay() inside a loop, so that the 1s delay is really 1s.
 1.4.12.4  16-Sep-2007  xtraeme branches: 1.4.12.4.4;
Pull up following revision(s) (requested by bouyer in ticket #882):
sys/dev/ic/ahcisata_core.c: revision 1.5
sys/dev/ic/ahcisatareg.h: revision 1.2

Fix various error handling bugs:
- the value of the ATA error register would be computed wrongly, leading to
bogus error values reported to wd(4)
- the channel would not always be restarted after an error, so the next
command would not be handled by the controller
- a timeout condition would not be properly reported to wd(4), leading
to a short transfer instead of a reset/retry
these bugs would cause a AHCI SATA channel to be stalled (no more command
processed) after a "ID not found" or "Aborted command" error reported by the
drive.
 1.4.12.3  31-Aug-2007  pavel Pull up following revision(s) (requested by bouyer in ticket #746):
sys/dev/ic/ahcisata_core.c: revision 1.2
* Add bus_dmamap_sync calls for cmd structure.
* During initialization, use the right port index when setting up the
physical pointers for a port. Fixes issue with non-contigous ports.
Reviewed by Manuel.
 1.4.12.2  31-Aug-2007  pavel Pull up following revision(s) (requested by bouyer in ticket #678):
distrib/sets/lists/man/mi: revision 1.1001
doc/CHANGES: revision 1.853 via patch
share/man/man4/Makefile: revision 1.428
share/man/man4/jmide.4: revision 1.1
sys/arch/amd64/conf/GENERIC: revision 1.143
sys/arch/amd64/conf/INSTALL: revision 1.69
sys/arch/i386/conf/GENERIC: revision 1.827
sys/arch/i386/conf/INSTALL_LARGE: revision 1.2 via patch
sys/arch/i386/conf/XEN2_DOM0: revision 1.26
sys/conf/files: revision 1.844 via patch
sys/dev/ic/ahcisata_core.c: revision 1.1 via patch
sys/dev/ic/ahcisata_core.c: revision 1.3
sys/dev/ic/ahcisatareg.h: revision 1.1
sys/dev/ic/ahcisatavar.h: revision 1.1
sys/dev/pci/ahcisata.c: delete
sys/dev/pci/ahcisatareg.h: delete
sys/dev/pci/ahcisata_pci.c: revision 1.1
sys/dev/pci/files.pci: revision 1.287-1.288
sys/dev/pci/jmide.c: revision 1.1-1.2
sys/dev/pci/jmide_reg.h: revision 1.1
sys/dev/pci/pcidevs: revision 1.878
Add JMicron Technology vendor ID, and their current PCIe SATA/PATA
controllers.

Split the ahcisata driver in pci front-end and bus-independant back-end.
add jmide(4), a driver for the JMicron Technology JMB36x PCIe to SATA II/PATA
controllers. These controllers can be found on add-on PCIe cards, or
on some motherboards to provide the PATA connectivity (e.g. some intel
ICH8-based motherboards).
Thanks to JMicron Technology for providing me documentation and
different sample boards for this work.

Move mapping of AHCI register so that jmide knows if it fails, and avoids
calling ahci_intr() (which would cause a panic).
Try to use the pciide function for SATA drives if attaching ahci fails (this
doesn't seems to work though, it may be BIOS dependant).
Thanks to Gary Duzan for testing multiple pacthes.

We're not ready to handle ATAPI yet so just claim there's no drive to the
upper layer. This should work around a NULL pointer dereference when
an ATAPI device is detected on a AHCI device.
 1.4.12.1  09-Jul-2007  pavel file ahcisata_core.c was added on branch netbsd-4 on 2007-08-31 20:09:23 +0000
 1.4.12.4.4.1  26-Feb-2009  snj Pull up following revision(s) (requested by bouyer in ticket #1277):
sys/dev/ic/ahcisata_core.c: revision 1.19
When an error is reported on a write, data may have been transfered
to the device's cache anyway and so cmdh_prdbc reports a completed
transfer. If we use it to update ata_bio->bcount this has 2 conseqences:
- the automatic LBA48 workaround doesn't qick in because bcount is used
to compute the last sector of the transfer (wd(4) part of kern/40569)
- wd(4) will report a B_ERROR buffer with a b_resid of 0, which panics
a DIAGNOSTIC kernel
Fix by ignoring cmdh_prdbc if we had a write with errors, and in this case
leave ata_bio->bcount at its initial value.
While there use NOERROR instead of 0 for ata_bio->error (cosmetic).
thanks to Matthias Scheler for tests.
 1.4.10.3  23-Mar-2008  matt sync with HEAD
 1.4.10.2  09-Jan-2008  matt sync with HEAD
 1.4.10.1  06-Nov-2007  matt sync with HEAD
 1.4.8.4  09-Dec-2007  jmcneill Sync with HEAD.
 1.4.8.3  14-Nov-2007  joerg Sync with HEAD.
 1.4.8.2  02-Oct-2007  joerg Sync with HEAD.
 1.4.8.1  04-Aug-2007  he Factor out the hardware initialization code in ahcisata_core.c,
and provide them to the pci frontend, which has now grown its own
softc and a power management handler.

There may be one too many re-initializations done on resume (witness
the multiple kernel messages from ahcisata), but at least this is
sufficient to get the Lenovo T60 to come out of resume with a
working disk controller and disk.
 1.4.2.2  11-Jul-2007  mjf Sync with head.
 1.4.2.1  09-Jul-2007  mjf file ahcisata_core.c was added on branch mjf-ufs-trans on 2007-07-11 20:05:37 +0000
 1.5.6.3  18-Feb-2008  mjf Sync with HEAD.
 1.5.6.2  08-Dec-2007  mjf Sync with HEAD.
 1.5.6.1  19-Nov-2007  mjf Sync with HEAD.
 1.5.4.1  13-Nov-2007  bouyer Sync with HEAD
 1.12.6.5  05-Oct-2008  mjf Sync with HEAD.
 1.12.6.4  28-Sep-2008  mjf Sync with HEAD.
 1.12.6.3  29-Jun-2008  mjf Sync with HEAD.
 1.12.6.2  02-Jun-2008  mjf Sync with HEAD.
 1.12.6.1  03-Apr-2008  mjf Sync with HEAD.
 1.12.2.1  24-Mar-2008  keiichi sync with head.
 1.14.4.5  11-Aug-2010  yamt sync with head.
 1.14.4.4  11-Mar-2010  yamt sync with head
 1.14.4.3  20-Jun-2009  yamt sync with head
 1.14.4.2  04-May-2009  yamt sync with head.
 1.14.4.1  16-May-2008  yamt sync with head.
 1.14.2.2  17-Jun-2008  yamt sync with head.
 1.14.2.1  18-May-2008  yamt sync with head.
 1.15.2.3  10-Oct-2008  skrll Sync with HEAD.
 1.15.2.2  18-Sep-2008  wrstuden Sync with wrstuden-revivesa-base-2.
 1.15.2.1  23-Jun-2008  wrstuden Sync w/ -current. 34 merge conflicts to follow.
 1.16.4.1  19-Oct-2008  haad Sync with HEAD.
 1.18.8.2  23-Jul-2009  jym Sync with HEAD.
 1.18.8.1  13-May-2009  jym Sync with HEAD.

Commit is split, to avoid a "too many arguments" protocol error.
 1.18.4.5  16-Jan-2011  bouyer Pull up following revision(s) (requested by tsutsui in ticket #1530):
sys/dev/ic/ahcisata_core.c: revision 1.31
Don't call bus_dmamap_load(9) and bus_dmamap_sync(9) on command xfers
if (AT_READ|AT_WRITE) in ata_c->flags is set but ata_c->bcount is zero.
Someone actually tries to put such a command and it causes
DIAGNOSTIC panic in x86/bus_dma.c:_bus_dmamap_sync().
I think bus_dma(9) API itself may allow calls with mapsize==0
but there are many MD code that asserts offset>=mapsize or len==0.
The problem is reported and fix is confirmed by Takuro KUBOTA
with XEN DOM0 kernel (which has options DIAGNOSTIC).
 1.18.4.4  28-Mar-2010  snj branches: 1.18.4.4.4;
Pull up following revision(s) (requested by bouyer in ticket #1317):
sys/dev/ic/ahcisata_core.c: revision 1.23, 1.24
Move ahci_channel_start() after BSY has cleared (what seems to matter
here is not touching SError while BSY is asserted). Fix random
behavior (serial console corruption/misfunction, misbehavior of USB
controllers, crasy interrupts crashing the Xen hypervisor, ...)
of Intel ESB2 controller with some brand of hitachi drives.
--
ahcisata_core.c 1.23 cause long drive probe (31s, reported by
Roy Marples), or other devices misbehavior probably due to interrupts issues
(reported by Jukka Ruohonen). Back it out and do the following changes:
- clear port interrupt register before ahci_channel_start() which enables
interrupts
- wait 500ms after sata_reset_interface() before touching SERROR register.
This is what seems to fix the issue I'm seeming on ESB2 controller.
- The 31s delay didn't cause the probe to fail because of a mismatch
in loop index comparison; use a #define for delay after reset
instead of numeric values, to avoid this kind of bugs in the
future.
 1.18.4.3  07-Mar-2010  sborrill Backout ticket 1317 until problems reported in -current are fixed:
http://mail-index.netbsd.org/current-users/2010/03/04/msg012810.html
 1.18.4.2  06-Mar-2010  sborrill Pull up the following revisions(s) (requested by bouyer in ticket #1317):
sys/dev/ic/ahcisata_core.c: revision 1.23

Move ahci_channel_start() after BSY has cleared (do not touch SError while
BSY is asserted). Fixes random behaviour of Intel ESB2 controller with some
models of Hitachi drives (serial console corruption/misfunction, misbehavior
of USB controllers, crazy interrupts crashing the Xen hypervisor, etc.).
 1.18.4.1  19-Feb-2009  snj branches: 1.18.4.1.4;
Pull up following revision(s) (requested by bouyer in ticket #455):
sys/dev/ic/ahcisata_core.c: revision 1.19
When an error is reported on a write, data may have been transfered
to the device's cache anyway and so cmdh_prdbc reports a completed
transfer. If we use it to update ata_bio->bcount this has 2 conseqences:
- the automatic LBA48 workaround doesn't qick in because bcount is used
to compute the last sector of the transfer (wd(4) part of kern/40569)
- wd(4) will report a B_ERROR buffer with a b_resid of 0, which panics
a DIAGNOSTIC kernel
Fix by ignoring cmdh_prdbc if we had a write with errors, and in this case
leave ata_bio->bcount at its initial value.
While there use NOERROR instead of 0 for ata_bio->error (cosmetic).
thanks to Matthias Scheler for tests.
 1.18.4.4.4.1  14-Oct-2011  matt revision 1.25
date: 2010/03/12 19:03:14; author: jakllsch; state: Exp; lines: +13 -2
ata(4) expects IDENTIFY data to be in host endianess.
Logic borrowed from siisata(4) (which I've confirmed works on sparc64).
 1.18.4.1.4.2  24-Mar-2014  matt Merge needed changes from HEAD for cubie
 1.18.4.1.4.1  21-Apr-2010  matt sync to netbsd-5
 1.18.2.1  03-Mar-2009  skrll Sync with HEAD.
 1.22.2.2  17-Aug-2010  uebayasi Sync with HEAD.
 1.22.2.1  30-Apr-2010  uebayasi Sync with HEAD.
 1.25.2.2  05-Mar-2011  rmind sync with head
 1.25.2.1  30-May-2010  rmind sync with head
 1.32.6.2  29-Apr-2012  mrg sync to latest -current.
 1.32.6.1  18-Feb-2012  mrg merge to -current.
 1.32.2.4  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.32.2.3  30-Oct-2012  yamt sync with head
 1.32.2.2  23-May-2012  yamt sync with head.
 1.32.2.1  17-Apr-2012  yamt sync with head
 1.33.4.1  03-Dec-2012  matt Pullup changes in rev 1.44.
 1.43.2.6  03-Dec-2017  jdolecek update from HEAD
 1.43.2.5  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.43.2.4  23-Jun-2013  tls resync from head
 1.43.2.3  25-Feb-2013  tls resync with head
 1.43.2.2  20-Nov-2012  tls Resync to 2012-11-19 00:00:00 UTC
 1.43.2.1  09-Oct-2012  bouyer Support transfers of up to MACHINE_MAXPHYS in all pciide variants, and ahci.
wd(4) limits its maxphys depending on the drives's capability (64k sectors
for LBA48, 256 sectors for LBA and 128 sectors for older devices).

I assumed all pciide controllers could do MACHINE_MAXPHYS transfers, but
this may not be true. The capabilities of each controller variants should be
looked at more closely.
 1.48.2.1  18-May-2014  rmind sync with head
 1.52.2.5  09-Jul-2016  skrll Sync with HEAD
 1.52.2.4  29-May-2016  skrll Sync with HEAD
 1.52.2.3  27-Dec-2015  skrll Sync with HEAD (as of 26th Dec)
 1.52.2.2  06-Jun-2015  skrll Sync with HEAD
 1.52.2.1  06-Apr-2015  skrll Sync with HEAD
 1.57.6.29  19-Sep-2017  jdolecek replace all remaining tsleep()/wakeup() calls with condition variables, or
calls to ata_delay(), as appropriate; change ata_delay() to require the
channel lock on entry, and pass the lock to kpause() for unlocking while
sleeping
 1.57.6.28  19-Sep-2017  jdolecek seems the CMD/CCS slot is always zero at least under QEMU for successful
polled commands, so go back to using it only on error path; while the value
seems good on real hardware, there is no good reason for register read anyway
 1.57.6.27  10-Sep-2017  jdolecek refactor code so that xfer c_start() hook is called with channel mutex held,
and hence the controller submit code no longer relies on spl

tested all the affected drivers - wdc (via piixide), ahci, mvsata, siisata,
both disk and atapi I/O
 1.57.6.26  12-Aug-2017  jdolecek do not reset drive after successful NCQ error recovery
 1.57.6.25  01-Aug-2017  jdolecek fix logic bug in processing of finished commands - mask of active
commands can change during the loop as c_intr() callback can queue
new commands, so the interrupt routine should only mark as finished
those which were actually active before the loop started; otherwise
the code marked as finished commands which were just started, and
being executed by HBA, leading to all sorts of data corruption

while here mark the active mask volatile, as it is modified from
interrupt context

this fixes for good the random crashes, short reads, and fatal command
errors which I've been tracing down for past couple weeks

thanks to Jonathan (jakllsch@) for testing, and a script to easily
triggered the condition, and led to this bug being finally found and squashed
 1.57.6.24  29-Jul-2017  jdolecek whitespace
 1.57.6.23  29-Jul-2017  jdolecek make compile without AHCI_DEBUG
 1.57.6.22  29-Jul-2017  jdolecek make ahci_channel_recover() non-static, so that it's visible in backtrace,
and can set a separate breakpoint there
 1.57.6.21  29-Jul-2017  jdolecek do not do the drive reset on non-fatal recovery - spec explicitely says
says the READ LOG EXT should only be done when neither COMRESET nor software
reset was done, and indeed it returns junk in this case

use C_RECOVERY slot for drive reset, so that it now will always succeed
in getting a slot, in cases when it would be necessary

adjust code and comments on the recovery path to explain better what's going on

with this AHCI error recovery works with real hardware without timeouts again
 1.57.6.20  23-Jul-2017  jdolecek rework the error handling and recovery, so that errors during the recovery
are handled correctly, and the recovery more closely follows the spec

this fixes e.g. NCQ error handling under QEMU, which doesn't implement
READ LOG EXT - previous code actually made the call 'succeed', returning bogus
(zero) slot/error/status

finished xfers are still handled before entering recovery (with channel frozen)
to avoid unnecessary retries
 1.57.6.19  21-Jul-2017  jdolecek use free slot for drive reset, rather than always using slot zero; if we can't
get the slot, fallback to channel reset as usual

note this increases the odds of not being able to do the reset, when all slots
happen to be active

this is in same area as problem reported by PR kern/52372 but I
don't believe that this change actually make any change for it - during
probe/attach there shouldn't be any paralell request with drive reset
 1.57.6.18  19-Jul-2017  jdolecek update error handling:
- switch to ata_timeout()
- stop using ch_status/ch_error for passing state/error, stop setting
ATACH_IRQ_WAIT in ch_flags; pass the state via the last parameter
to c_intr() routine
- add NCQ recovery and KILL_REQUEUE
- only call atastart() in c_intr() if there was no error

ahcisata-specific tweaks:
- add some handling for PM in the error recovery using FBS register,
according to spec it should be independant of actual FBSS feature; untested
as my hw doesn't support PM
 1.57.6.17  27-Jun-2017  jdolecek attend error paths, more strict asserts and code consistency

- atastart() and ata_kill_pending() now KASSERT() that all xfers on queue
have same channel
- inactive xfers are killed via new reason KILL_GONE_INACTIVE, controller
code must not call any resource deactivation in that case
- c_intr() must call ata_waitdrain_xfer_check() as first thing, and must not
further touch any xfer structures on exit path; any resource cleanup
is supposed to be done in c_kill_xfer()
- c_kill_xfer() should never call atastart()
- ata_waitdrain_check() removed, replaced by ata_waitdrain_xfer_check()
- ATA_DRIVE_WAITDRAIN handling converted to use condvar
- removed unused ata_c callback
 1.57.6.16  21-Jun-2017  jdolecek change ata_queue_hwslot_to_xfer() and ata_queue_get_active_xfer() to take
ata_channel instead of ata_queue as parameter, and lock the channel while
traversing the queue
 1.57.6.15  20-Jun-2017  jdolecek remove ata queue downsizing - every device, attached to the same channel,
uses slots according to it's own limits

wdc code changed to expect maximum one active xfer, and not check number
of openings in the channel; this is to facilitate using wdc functions
for e.g. handling of atapi commands for drivers which support both ATAPI
and NCQ
 1.57.6.14  19-Jun-2017  jdolecek add ata_channel lock, use it to protect queue manipulation (only that for now);
add ata_channel_detach() to destroy the locks

change ata_get_xfer() so that it can wait for xfer, convert all on-stack
xfer code to use the blocking variant

fix siisata_reset_drive() to use polled reset and not try ata_activate_xfer(),
convert drive probe code also over from slot0 XXX to ata_get_xfer()

drive reset and PMP now works on siisata(4) too; changes tested also
on piixide(4), ahci(4), mvsata(4)
 1.57.6.13  16-Jun-2017  jdolecek adjust reset channel and dump paths
- channel reset now always kills active transfer, even on dump path, but
now doesn't touch the queued waiting transfers; also kill_xfer hook is
always called, so that HBA can free any private xfer resources and thus
the dump request has chance to work
- kill_xfer routines now always call ata_deactivate_xfer(); added KASSERT()s
to ata_free_xfer() to expect deactivated xfer
- when called during channel reset before dump, ata_kill_active() drops
any queued waiting transfers without processing
- do not (re)queue any transfers in wddone() when dumping
- kill AT_RST_NOCMD flag

This should also hopefully fix the 'polled command has been queued' panic
as reported in:
PR kern/11811 by John Hawkinson
PR kern/47041 by Taylor R Campbell
PR kern/51979 by Martin Husemann

dump tested working with piixide(4) and ahci(4). mvsata(4) dump times out,
but otherwise tested working, will be fixed separately. siisata(4) mechanically
changed and not tested.
 1.57.6.12  25-Apr-2017  jdolecek make compile without DIAGNOSTIC
 1.57.6.11  24-Apr-2017  jdolecek only clear the bcount for NCQ case when the transfer actually ended w/o error
 1.57.6.10  24-Apr-2017  jdolecek do not use PRD Byte Count for queued commands to determine number of bytes
actually transferred; it should not be used and is not guaranteed to be
valid according to AHCI spec, underflow is always invalid for queued cmds

this fixes my ls problem under QEMU
 1.57.6.9  24-Apr-2017  jdolecek adjust error handling in interrupt handler - finish the successfully
finished xfers, and iterate to error out all the unfinished ones
 1.57.6.8  20-Apr-2017  jakllsch Don't bail out of handling interrupts at the first inactive slot.

Appears to fix the frequent timeouts issue.
 1.57.6.7  19-Apr-2017  jdolecek adjust ata code to support more than one active command, including the
timeout handling, add support for NCQ commands

move probe for NCQ and number of tags to middle layer, negotiate mutual
support between drive and controller

implement NCQ support in ahci(4)
 1.57.6.6  15-Apr-2017  jdolecek use xfer->c_slot instead of slot = 0

fix ahci_bio_complete() calls - last param is reason, not slot
 1.57.6.5  15-Apr-2017  jdolecek ahci_intr_port(): use same code on both the error and non-error path for
for getting the slot and skipping interrupts for non-active slots
 1.57.6.4  15-Apr-2017  jdolecek pass also ata_command via ata_xfer, callers of ata_exec_command() is now
responsible for allocation/disposal of the structure

change code to allocate ata_xfer for commands on stack same way as previously
the ata_command were, using c_slot 0; adjust asserts so that it would allow
several xfers with same c_slot, as long as only one such transfer is active
at a time
 1.57.6.3  15-Apr-2017  jdolecek make ata_xfer's allocated as part of ata_queue and make it include ata_bio;
they are pre-allocated on attach and ata_get_xfer() now never sleep, drop the
pool

modify wd(4) to file the bio requests using the xfers and hence
make it possible to have more than one active I/O request in flight;
ata_bio callback doesn't need to allocate any memory any more,
require it to never return ATACMD_TRY_AGAIN

move lp, badsect, multi from ata_bio to ata_drive_datas, as they are per-drive,
not per transfer

drop unused drv atac_claim_hw/atac_free_hw hooks, and also drop again ata_bio
c_hwslot
 1.57.6.2  11-Apr-2017  jdolecek fix to not access active_xfer directly
 1.57.6.1  10-Apr-2017  jdolecek ATA infrastructure improvements to eventually support more outstanding
commands

patch by Matt Thomas
 1.60.4.4  21-Apr-2020  martin Sync with HEAD
 1.60.4.3  13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.60.4.2  08-Apr-2020  martin Merge changes from current as of 20200406
 1.60.4.1  10-Jun-2019  christos Sync with HEAD
 1.60.2.6  26-Jan-2019  pgoyette Sync with HEAD
 1.60.2.5  18-Jan-2019  pgoyette Synch with HEAD
 1.60.2.4  26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.60.2.3  26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.60.2.2  30-Sep-2018  pgoyette Ssync with HEAD
 1.60.2.1  28-Jul-2018  pgoyette Sync with HEAD
 1.62.2.10  15-Oct-2018  jdolecek change the SATA/NCQ recovery to run in the atabus thread
 1.62.2.9  11-Oct-2018  jdolecek refactor shared parts of the SATA error recovery into new function
ata_recovery_resume() and use for ahcisata/siisata/mvsata, also replace
per-controller hold/unhold with generic version

move the shared recovery code into separate file ata_recovery.c
 1.62.2.8  07-Oct-2018  jdolecek move atastart() call after xfer deactivation during ATA command completion,
so that it's executed after the slot is freed
 1.62.2.7  04-Oct-2018  jdolecek further channel locking pass for reset
 1.62.2.6  03-Oct-2018  jdolecek change channel reset and drive reset for all ATA controllers to always
run via thread, and with channel lock held the whole time; the queue is
frozen while reset is pending

for this repurpose ata_reset_channel() into new ata_thread_run()

also adjust some device printfs to not leak xfer pointer, and avoid
aprint_* for non-autoconf messages
 1.62.2.5  22-Sep-2018  jdolecek separate ata_xfer slot allocation and the memory allocation, so that
there can be more queued xfers than number of supported slots by controller,
and use a pool instead of custom pre-allocation

primarily to help PR kern/52614

remove no longer needed custom wd(4) logic for flush cache

switch also wd(4) trim/suspend/setcache/wdioctlstrategy to sleep waiting
for the memory, they are all called from process context and this
avoids spurious failures
 1.62.2.4  17-Sep-2018  jdolecek move ATAPI-only members of ata_xfer to an union struct to further save space
 1.62.2.3  17-Sep-2018  jdolecek move low-level protocol handlers hooks from ata_xfer to separate struct,
initialized statically

primarily to reduce ata_xfer struct size, but also improves readibility,
and enforces consistency
 1.62.2.2  01-Sep-2018  jdolecek only call ata_deactivate_xfer() once completely done with the active
xfer in controller code (i.e. after bus_dmamap_sync() et.al.), so that
the command slot is safe to be reused immediatelly after deactivate
 1.62.2.1  31-Aug-2018  jdolecek refactor ata_xfer to be just dumb structure; move all callouts/condvars out

retry callout to wd(4); reset callout and the active/cmd finish condvars
to channel queue; change code using the condvars so it works if there
are multiple waiters

simplify the async wait code for cmds, replace ata_wait_xfer()/ata_wake_xfer()
with ata_wait_cmd()

fix the callout_invoking/ack race handling code for timeouts to
actually have chance to work; change mvsata(4) to use generic timeout func

towards resolution of kern/52614
 1.75.4.5  30-Dec-2022  martin Pull up following revision(s) (requested by tsutsui in ticket #1557):

sys/dev/ic/ahcisata_core.c: revision 1.83
sys/dev/ic/ahcisata_core.c: revision 1.102
sys/dev/ata/ata.c: revision 1.164
sys/dev/ata/ata_wdc.c: revision 1.115
sys/dev/ata/ata_recovery.c: revision 1.4
sys/dev/ic/siisata.c: revision 1.42
sys/dev/ic/wdc.c: revision 1.308
sys/dev/ic/mvsata.c: revision 1.56
sys/dev/scsipi/atapi_wdc.c: revision 1.138
sys/dev/ic/siisata.c: revision 1.49
sys/dev/ata/atavar.h: revision 1.105
sys/dev/ata/wd.c: revision 1.460
sys/dev/ata/ata.c: revision 1.155
sys/dev/ata/wd.c: revision 1.462
sys/dev/ata/atavar.h: revision 1.109
sys/dev/ata/satapmp_subr.c: revision 1.16
sys/dev/ic/wdc.c: revision 1.299
sys/dev/ic/ahcisata_core.c: revision 1.93
sys/dev/ata/ata_wdc.c: revision 1.120
sys/dev/ic/wdcvar.h: revision 1.100
sys/dev/scsipi/atapi_wdc.c: revision 1.141
sys/dev/ic/mvsata.c: revision 1.61
sys/dev/usb/umass_isdata.c (apply patch)

drop wd lock in wdstart1() before calling the ata_bio hook; when called
from ata thread context, that can still need to sleep for wdc attachments
in wdcwait()

fix use-after-free for ata xfer on bio submission found by KASAN
driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself
PR kern/55169 by Nick Hudson

Function declaration formating whitespace consistency. NFCI.

PR kern/56403
Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.
(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.
Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().
(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).
"Go ahead" by jdolecek@.
 1.75.4.4  30-Dec-2020  martin Pull up following revision(s) (requested by jmcneill in ticket #1167):

sys/dev/ic/ahcisata_core.c: revision 1.84
sys/dev/ic/ahcisata_core.c: revision 1.85
sys/dev/ic/ahcisata_core.c: revision 1.88
sys/dev/ic/ahcisata_core.c: revision 1.89
sys/arch/arm/nvidia/tegra_ahcisata.c: revision 1.13
sys/dev/ic/ahcisatavar.h: revision 1.26
sys/dev/ic/ahcisata_core.c: revision 1.90
sys/dev/ic/ahcisata_core.c: revision 1.91
sys/dev/ic/ahcisata_core.c: revision 1.92
sys/dev/ata/satareg.h: revision 1.6

ahci_exec_fis: wait for the correct amount of time when AT_WAIT is set

Retry clearing WDCTL_RST a few times before giving up. Makes SATA work in
Solidrun Honeycomb LX2K.

AHCI 1.3.1 specification says that it is good practice for system software
to 'zero-out' the memory allocated and referenced by PxCLB and PxFB.

ahci_intr: use ffs in the port bitmask instead of looping over all 32 bits

AHCI 1.3.1 section 5.5.3 "Processing Completed Commands" says that we
should clear PxIS before IS.IPS.

Add G3 and DevSleep definitions. This changes the mask used by
SControl_IPM_NONE from 0x3 to 0x7.

Make sure to ack IS after PxIS when polling and when using multiple MSI-X
messages.

Remove the AHCI_QUIRK_SKIP_RESET quirk now that the underlying issue is
fixed.
 1.75.4.3  21-Jan-2020  martin Pull up following revision(s) (requested by simonb in ticket #630):

sys/dev/ic/ahcisatavar.h: revision 1.24
sys/dev/pci/ahcisata_pci.c: revision 1.57
sys/dev/ic/ahcisata_core.c: revision 1.81

Add new AHCI_QUIRK_BADNCQ quick for controllers that have issues with
NCQ on (some) drives. Enable this quirk for ATI (AMD) SB600/SB700
controllers. Alternate fix for kern/54790 and kern/54855.
ok jdolecek@, tested on my SB700 chipset and tsutsui's SB600 chipset.
 1.75.4.2  24-Dec-2019  martin Pull up following revision(s) (requested by jmcneill in ticket #576):

sys/dev/ic/ahcisata_core.c: revision 1.79

When resetting a drive, if the command list is running and CLO is not
supported, attempt to stop the drive first and fail gracefully if that
fails instead of triggering a KASSERT on DIAGNOSTIC kernels.
 1.75.4.1  23-Oct-2019  martin Pull up following revision(s) (requested by jdolecek in ticket #357):

sys/dev/ic/ahcisata_core.c: revision 1.76

Ensure cmdh_prdtl is 0 in ahci_do_reset_drive().
(I'm probably just being paranoid here.)
 1.80.2.2  29-Feb-2020  ad Sync with head.
 1.80.2.1  25-Jan-2020  ad Sync with head.
 1.82.4.1  20-Apr-2020  bouyer Sync with HEAD
 1.83.2.2  03-Apr-2021  thorpej Sync with HEAD.
 1.83.2.1  03-Jan-2021  thorpej Sync w/ HEAD.
 1.97.2.1  02-Apr-2021  thorpej config_found_ia() -> config_found() w/ CFARG_IATTR.
 1.98.2.1  01-Aug-2021  thorpej Sync with HEAD.
 1.99.2.1  04-Aug-2021  thorpej Adapt to CFARGS().
 1.107.4.1  11-Sep-2023  martin Pull up following revision(s) (requested by abs in ticket #366):

sys/dev/pci/ahcisata_pci.c: revision 1.70
sys/dev/ic/ahcisata_core.c: revision 1.108
sys/dev/ic/ahcisatavar.h: revision 1.28
sys/conf/files: revision 1.1309

Rework AHCISATA_EXTRA_DELAY for kern/56737
- Remove AHCI_QUIRK_EXTRA_DELAY as issue appears to be drive and
not controller related
- Replace AHCISATA_EXTRA_DELAY with AHCISATA_REMOVE_EXTRA_DELAY,
so defaulting to enabling the extra delay, as the downside of
slower probing on systems which do not need it is less than having
other systems intermittently fail to probe and attach drives
- Also allow disabling extra delay with AHCISATA_EXTRA_DELAY_MS = 0

We should return to this code to work out which of the extra delays
are needed, and how long they need to be. It may be that faster
systems are more likely to trigger the issue (I've only seen it on
a 13th gen i7-13700, though only tested on a limited set)

RSS XML Feed