Home | History | Annotate | Download | only in ic
History log of /src/sys/dev/ic/mvsata.c
RevisionDateAuthorComments
 1.62  05-Dec-2021  msaitoh s/coresponding/corresponding/ in comment.
 1.61  05-Oct-2021  rin PR kern/56403

Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.

(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.

Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().

(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).

"Go ahead" by jdolecek@.
 1.60  07-Aug-2021  thorpej Merge thorpej-cfargs2.
 1.59  24-Apr-2021  thorpej branches: 1.59.8;
Merge thorpej-cfargs branch:

Simplify and make extensible the config_search() / config_found() /
config_attach() interfaces: rather than having different variants for
which arguments you want pass along, just have a single call that
takes a variadic list of tag-value arguments.

Adjust all call sites:
- Simplify wherever possible; don't pass along arguments that aren't
actually needed.
- Don't be explicit about what interface attribute is attaching if
the device only has one. (More simplification.)
- Add a config_probe() function to be used in indirect configuiration
situations, making is visibly easier to see when indirect config is
in play, and allowing for future change in semantics. (As of now,
this is just a wrapper around config_match(), but that is an
implementation detail.)

Remove unnecessary or redundant interface attributes where they're not
needed.

There are currently 5 "cfargs" defined:
- CFARG_SUBMATCH (submatch function for direct config)
- CFARG_SEARCH (search function for indirect config)
- CFARG_IATTR (interface attribte)
- CFARG_LOCATORS (locators array)
- CFARG_DEVHANDLE (devhandle_t - wraps OFW, ACPI, etc. handles)

...and a sentinel value CFARG_EOL.

Add some extra sanity checking to ensure that interface attributes
aren't ambiguous.

Use CFARG_DEVHANDLE in MI FDT, OFW, and ACPI code, and macppc and shark
ports to associate those device handles with device_t instance. This
will trickle trough to more places over time (need back-end for pre-OFW
Sun OBP; any others?).
 1.58  25-Dec-2020  skrll branches: 1.58.2;
Use designated initializers for struct ata_bustype
 1.57  19-May-2020  jdolecek branches: 1.57.2;
only start the timeout machinery once the I/O is completely setup
and successful, particularly after PIO write is finished

fixes crashes in case the setup is so slow that timeout is triggered
e.g. while still waiting in wdc_wait_for_unbusy() or shortly after, without
drive actually having chance to complete the I/O, as seen in some
configuration under QEMU by Paul Ripke
 1.56  13-Apr-2020  jdolecek fix use-after-free for ata xfer on bio submission found by KASAN

driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself

PR kern/55169 by Nick Hudson
 1.55  04-Apr-2020  jdolecek branches: 1.55.2;
fix deadlock in wdcwait() when xfer timeout happens while the atabus
thread sleeps in wdcwait() - check current lwp rather than relying
on global ATACH_TH_RUN channel flag

should fix the hang part of the problem reported in
http://mail-index.netbsd.org/netbsd-users/2020/03/12/msg024249.html

thanks to Paul Ripke for providing extensive debugging info
 1.54  22-Mar-2020  macallan do not assume that a bus_space_handle_t is simply an offset
now this has a fighting chance of working on sparc64
 1.53  19-Feb-2020  riastradh C99 initializers for scsipi_bustype. No functional change intended.
 1.52  27-Dec-2019  msaitoh branches: 1.52.2;
s/transfered/transferred/
 1.51  22-Dec-2019  jdolecek don't treat the EDMA self disable as autoconfig error, it's perfectly
normal at least during boot when device connection is reported by controller
 1.50  22-Dec-2019  jdolecek remove check for idle status when disabling EDMA, and always toggle
the disable regardless of current state - this particularly seems
to fail during error recovery, and on my system this also fails
during regular boot

this matches both FreeBSD and Linux drivers - neither of those checks
the idle status

should help with PR kern/52419 and maybe also the lock spinout part
of PR kern/52126
 1.49  10-Nov-2019  chs in many device attach paths, allocate memory with M_WAITOK instead of M_NOWAIT
and remove code to handle failures that can no longer happen.
 1.48  23-Jun-2019  tsutsui branches: 1.48.2;
Add a missing newline in an error message. Noticed in PR/54205
 1.47  23-Jun-2019  tsutsui Fix "mvsata:0:0:1: buffer load failed: error=27" failure. PR/54205

Ok'ed by jdolecek@.
Should be pulled up to netbsd-8.
 1.46  12-Nov-2018  jdolecek pass correct status + error to *_atapi_phase_complete(), so that
the function is actually able to recognize when there was an error;
tested via reading a cd0 device in QEMU with ejected cdrom

bug was introduced with jdolecek-ncq branch

fixes PR kern/53724 by Andreas Gustafsson
 1.45  12-Nov-2018  jdolecek hold channel lock during whole ata_dmaerr()/ata_downgrade_mode() -
according to code inspection this is safe, none of the set_modes
hooks execute anything which would be taking the lock

adresses PR kern/53714 by Andreas Gustafsson
 1.44  22-Oct-2018  jdolecek move mvsata(4) related items from ata/TODO.ncq to dev/ic/mvsata.c, it's
unlikely this will ever be done, but still good to note somewhere
 1.43  22-Oct-2018  jdolecek Merge jdolecek-ncqfixes branch

- ata_xfer's are dynamicall allocated as needed using a pool, no longer
limited to number of possible openings supported by controller; dump
and recovery paths use dedicated pre-allocated storage
- moved callouts and condvars from ata_xfer to queue or channel, so that
ata_xfer does not need special initialization
- slot allocation now done when xfer is being activated, uncoupled
from memory allocation; active slots are no longer tracked by controller
code
- channel and drive reset is done always via the atabus thread, and
now executes with channel locked the whole time
- NCQ recovery moved to shared function, and run via the thread also
- added some workarounds for buggy error recovery AHCI emulation in QEMU
and Parallels

designed to primarily fix kern/52614, but might also help with kern/47041
and kern/53183
 1.42  03-Sep-2018  riastradh Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)
 1.41  31-Aug-2018  jdolecek branches: 1.41.2;
constify mvsata_pci_products[]
 1.40  08-Apr-2018  mlelstv branches: 1.40.2;
Avoid rounding errors for timeout values.
 1.39  17-Oct-2017  jdolecek branches: 1.39.2;
no need to clear DMA WAIT flag in mvsata_edma_handle(), it's not needed
by atabus layer
 1.38  10-Oct-2017  jdolecek make compile with ATAPI_DEBUG_PROBE (by copying over the code from
atapi_wdc.c), which is on e.g. for i386/amd64 ALL kernels
problem found by hannken@, thanks
 1.37  08-Oct-2017  joerg Initialize tfd even when not waiting for DSC.
 1.36  07-Oct-2017  jdolecek Merge support for SATA NCQ (Native Command Queueing) from jdolecek-ncq branch

ATA subsystem was changed to support several outstanding commands, and use
NCQ xfers if supported by both the controller and the disk, including NCQ
error recovery. Set NCQ high priority for BPRIO_TIMECRITICAL xfers
if supported. Added FUA support.

Done some work towards MP-safe, all ATA code tsleep()/wakeup() replaced
by condvars, and switched most code from spl* to mutexes (separate
wd(4) and ata channel lock).

Introduced new option WD_CHAOS_MONKEY to facilitate testing of error
handling, fixed several uncovered issues. Also fixed several problems
with kernel dump to wd(4) disk.

Tested with ahcisata(4), mvsata(4), siisata(4), piixide(4) on amd64,
with and without port multiplier, both disk and ATAPI devices; other
drivers and archs mechanically adjusted and compile-tested. NCQ is
supported for ahcisata(4) and siisata(4) for any controller, for
mvsata(4) only Gen IIe ones for now. Also enabled ATAPI support in
mvsata(4).

Thanks to Matt Thomas for initial ATA infrastructure patch, and
Jonathan A.Kollasch for siisata(4) NCQ changes and general testing.

Also fixes PR kern/43169 (wd(4)); and PR kern/11811, PR kern/47041,
PR kern/51979 (kernel dump)
 1.35  02-May-2016  christos branches: 1.35.6; 1.35.10;
move scsipi_strvis -> libkern:strnvisx()
change the prototype to match userland
fix sizes of strings passed to it
 1.34  02-Jan-2015  christos We have three sets of DTYPE_ constants in the kernel:
altq Drop Type
disklabel Disk Type
file Descriptor Type
(not to mention constants that contain the string DTYPE).
Let's make them two, by changing the disklabel one to be DisK TYPE since the
other disklabel constants seem to do that. Not many userland programs use
these constants (and the ones that they do are mostly in ifdefs). They will
be fixed shortly.
 1.33  17-Feb-2014  kiyohara branches: 1.33.6;
More delay() for some drives.
 1.32  01-Sep-2013  kiyohara Declare mvsata_probe_drive() when defined MVSATA_WITHOUTDMA.
And more delay() for some machines.
 1.31  06-May-2013  jakllsch branches: 1.31.4;
Put back AT_POLL hack in mvsata_bio_ready(). Without it 1.5Gbps
SATA WD800JD drives cause the kernel to stall during autoconf on
Orion.
 1.30  03-Apr-2013  bouyer Fix kernel dump on ahci controller, by making sure we won't sleep
while dumping:
- introduce ata_delay() which either use delay() or kpause()
depending on flags. use it in sata_reset_interface() and
some ahci functions
- kill ATA_NOSLEEP, it was tested but never set. use ATA_POLL instead.
- reduce delay while polling in ahci, to speed up the dump

Should fix PR kern/41095
 1.29  10-Feb-2013  jakllsch This change adds SATA port multiplier support to mvsata(4).
 1.28  10-Feb-2013  jakllsch Timeouts have priority over panicing.
 1.27  10-Feb-2013  jakllsch The device queue tag and the host queue tag are not the same thing.
We don't support device queueing yet, so write that tag as zero.
 1.26  10-Feb-2013  jakllsch We can't (well, actually we can, but we still can't) assume that all
ata_bios are LBA.
 1.25  03-Feb-2013  jakllsch Add argument to wdccommandext() to allow the entire contents of the
device/head register to be specified. Needed for upcoming port multipler
support in mvsata(4).
 1.24  31-Jul-2012  bouyer branches: 1.24.2;
Apply back changes that were reverted on Jul 24 and Jul 26 (general ata/wdc
cleanup and SATA PMP support), now that I'm back to fix the fallouts.
 1.23  26-Jul-2012  jakllsch Revert, with intention of restoring in a less invasive way, the SATA Port
Multiplier code.

ok christos@
 1.22  24-Jul-2012  jakllsch Revert dsl@'s changes of Sun, 15 Jul 2012 10:55:35 +0000 and
Sun, 15 Jul 2012 10:56:50 +0000, excepting the kernel version bump.
First step in reverting regressions to ata(4) subsystem during the addition of
port multiplier support.
 1.21  19-Jul-2012  jakllsch Fix MVSATA_WDC_* macros to actually access the Shadow Register Block.
 1.20  15-Jul-2012  dsl Some namespace protection (and add greppablity).
Prefix the DRIVE_ and DRIVET_ constants from atavar.h with ATA_.
Don't use an enum for drive_type - you don't know how big it will be.
Move driver_type to avoid implicit structure padding (esp on arm).
This change is purely lexical and mechanical.

Update to 6.99.9 - this wasn't done when the SATA PMP changes
were made - I'm sure they warranted a bump.
 1.19  15-Jul-2012  jakllsch Fix obvious copy/paste-induced error (my fault) that would clobber
the bottom 24-bits of r_lba during a LBA48 command with AT_READREG.
 1.18  12-Jul-2012  reinoud Fix surplus ( that was forgotten to remove in the DRIVE_ to DRIVET_ rototol.
 1.17  02-Jul-2012  bouyer Add sata Port MultiPlier (PMP) support to the ata bus layer,
as described in
http://mail-index.netbsd.org/tech-kern/2012/06/23/msg013442.html
PMP support in integrated to the atabus layer.
struct ata_channel's ch_drive[] is not dynamically allocated, and ch_ndrive
(renamed to ch_ndrives) closely reflects the size of the ch_drive[] array.
Add helper functions atabus_alloc_drives() and atabus_free_drives()
to manage ch_drive[]/ch_ndrives.
Add wdc_maxdrives to struct wdc_softc so that bus front-end can specify
how much drive they really support (master/slave or single).
ata_reset_drive() callback gains a uint32_t *sigp argument which,
when not NULL, will contain the signature of the device being reset.
While there, some cosmetic changes:
- added a drive_type enum to ata_drive_datas, and stop encoding the
probed drive type in drive_flags (we were out of drive flags anyway).
- rename DRIVE_ATAPIST to DRIVE_ATAPIDSCW to better reflect what this
really is
- remove ata_channel->ata_drives, it's redundant with the pointer in
ata_drive_datas
- factor out the interpretation of SATA signatures in sata_interpet_sig()

propagate these changes to the ATA HBA drivers, and add support for PMP
to ahcisata(4) and siisata(4).

Thanks to:
- Protocase (http://www.protocase.com/) which provided a system
with lots of controllers, SATA PMP and drive slots
- Conservation Genomics Laboratory, Department of Biology, New Mexico State
University for hosting the above system
- Brook Milligan, who set up remote access and has been very responsive
when SATA cable move was needed
 1.16  20-Apr-2012  bouyer Add a bustype_async_event_xfer_mode() callback to scsipi_bustype (which can
be NULL), so that transport-specific details of transfer mode setting/printing
can be handled more easily.
Move scsipi_async_event_xfer_mode() and scsipi_print_xfer_mode() to
scsi_base.c and split in parallel scsi and FC/SAS parts.
size of struct scsipi_bustype has changed, welcome to 6.99.5
 1.15  24-Jan-2012  jakllsch Rework struct ata_command to support LBA28 and LBA48-protocol commands.
Implement 28- and 48-bit command issuance and parameter read-back in the
various ATA host drivers. Add LBA28-protocol support to ATAIOCCOMMAND ioctl.
 1.14  22-Jan-2012  jakllsch Trailing whitespace.
 1.13  21-Jan-2012  jakllsch Correct some typos in comments, as was done in wdc.c 1.266.
 1.12  09-Jan-2012  jakllsch Instead of assume that 'features' is 0 in wdccommandext(),
pass it as an argument, as is done for wdccommand().
 1.11  01-Sep-2011  jakllsch branches: 1.11.2; 1.11.6;
It seems mvsata_bio_ready() is actually needed, and it seems
to work better if we always poll for completion for the
duration of this function.
 1.10  15-Aug-2011  jakllsch Even if the hardware can handle concurrent transfers, ata(4) can't.
Also, completely rework mvsata_bio_ready().

Seems to make mvsata(4) usable on 5182.
 1.9  14-Aug-2011  jakllsch Spell enqueue correctly.
 1.8  10-Mar-2011  jakllsch Shorten infinite loop. Tames atactl(8) atabus reset on mvsata(4).
 1.7  20-Feb-2011  riz Use PRId64 instead of lld, and PRIx64 instead of llx when compiling
with MVSATA_DEBUG.
 1.6  13-Jul-2010  kiyohara branches: 1.6.2; 1.6.4;
Fix wrong ifdef.
 1.5  13-Jul-2010  kiyohara Move to pci/mvsata_pci.c (marvell/mvsata_mv.c) splitting mvsata_products[].
mvsata(4) uses these to match.
 1.4  28-Mar-2010  snj Spell "enough" properly.
 1.3  05-Jan-2010  mbalmer branches: 1.3.2; 1.3.4;
Remove extra semicolons.
 1.2  03-Aug-2009  snj branches: 1.2.2;
Fix a typo inside a comment.
 1.1  27-Jul-2009  kiyohara Support Marvell Hercules-I/II SATA Controllers.
 1.2.2.4  11-Aug-2010  yamt sync with head.
 1.2.2.3  11-Mar-2010  yamt sync with head
 1.2.2.2  19-Aug-2009  yamt sync with head.
 1.2.2.1  03-Aug-2009  yamt file mvsata.c was added on branch yamt-nfs-mp on 2009-08-19 18:47:07 +0000
 1.3.4.3  21-Apr-2011  rmind sync with head
 1.3.4.2  05-Mar-2011  rmind sync with head
 1.3.4.1  30-May-2010  rmind sync with head
 1.3.2.2  17-Aug-2010  uebayasi Sync with HEAD.
 1.3.2.1  30-Apr-2010  uebayasi Sync with HEAD.
 1.6.4.1  05-Mar-2011  bouyer Sync with HEAD
 1.6.2.1  06-Jun-2011  jruoho Sync with HEAD.
 1.11.6.2  29-Apr-2012  mrg sync to latest -current.
 1.11.6.1  18-Feb-2012  mrg merge to -current.
 1.11.2.4  22-May-2014  yamt sync with head.

for a reference, the tree before this commit was tagged
as yamt-pagecache-tag8.

this commit was splitted into small chunks to avoid
a limitation of cvs. ("Protocol error: too many arguments")
 1.11.2.3  30-Oct-2012  yamt sync with head
 1.11.2.2  23-May-2012  yamt sync with head.
 1.11.2.1  17-Apr-2012  yamt sync with head
 1.24.2.4  03-Dec-2017  jdolecek update from HEAD
 1.24.2.3  20-Aug-2014  tls Rebase to HEAD as of a few days ago.
 1.24.2.2  23-Jun-2013  tls resync from head
 1.24.2.1  25-Feb-2013  tls resync with head
 1.31.4.1  18-May-2014  rmind sync with head
 1.33.6.2  29-May-2016  skrll Sync with HEAD
 1.33.6.1  06-Apr-2015  skrll Sync with HEAD
 1.35.10.1  30-Jun-2019  martin Pull up following revision(s) (requested by tsutsui in ticket #1283):

sys/dev/ic/mvsata.c: revision 1.47
sys/dev/ic/mvsata.c: revision 1.48

Fix "mvsata:0:0:1: buffer load failed: error=27" failure. PR/54205
Ok'ed by jdolecek@.
Should be pulled up to netbsd-8.

Add a missing newline in an error message. Noticed in PR/54205
 1.35.6.32  30-Sep-2017  jdolecek must drop channel lock before calling ata_dmaerr() to avoid 'locking against
myself' in case of consecutive errors from ata_reset_channel() called via
ata_downgrade_mode()
 1.35.6.31  27-Sep-2017  jdolecek change wdc_init_shadow_regs() to accept only struct wdc_regs, it doesn't
touch anything else

factor out the probe-only struct ata_channel initialization to wdcprobe(), to
reduce duplication of logic in individual drivers, and to actually work now
that more init is needed beyond the memset()
 1.35.6.30  26-Sep-2017  jdolecek make compile without MVSATA_DEBUG
 1.35.6.29  25-Sep-2017  jdolecek move mvsata_quetag_get() to the start routines, so that it always matches
the active list of ata queue; important during error recovery, fixes
panics when calling

in mvsata_edma_handle() ignore events for non-active xfers

fix some missing ata_channel_unlock() on error path in mvsata_bio_intr()
and mvsata_atapi_intr(), fixes 'locking against myself' with LOCKDEBUG

with this, mvsata(4) finally survives full fio run with wdcdebug_wd_cnt == 200
 1.35.6.28  22-Sep-2017  jdolecek fix inverted logic for calling atastart()
 1.35.6.27  20-Sep-2017  jdolecek make compile again with MVSATA_WITHOUTDMA
 1.35.6.26  19-Sep-2017  jdolecek replace all remaining tsleep()/wakeup() calls with condition variables, or
calls to ata_delay(), as appropriate; change ata_delay() to require the
channel lock on entry, and pass the lock to kpause() for unlocking while
sleeping
 1.35.6.25  10-Sep-2017  jdolecek refactor code so that xfer c_start() hook is called with channel mutex held,
and hence the controller submit code no longer relies on spl

tested all the affected drivers - wdc (via piixide), ahci, mvsata, siisata,
both disk and atapi I/O
 1.35.6.24  10-Sep-2017  jdolecek move mvsata_bio(), mvsata_exec_command() and mvsata_atapi_scsipi_request()
just before their respective hook functions
 1.35.6.23  13-Aug-2017  jdolecek put the non-NCQ KASSERT() before edma disable for bio PIO, we are not supposed
to get there with NCQ command even on retries any more
 1.35.6.22  12-Aug-2017  jdolecek add NCQ error recovery for mvsata(4)

fix wait flags for mvsata_bio_start(), AT_* constants are only valid for cmd

change the debug logging to similar form as ahcisata/siisata, so it's
easier to selectively show
 1.35.6.21  12-Aug-2017  jdolecek convert the atabus thread to use the channel lock and a condvar, adjust
code which sets the relevant channel flags to take the lock while doing so
 1.35.6.20  12-Aug-2017  jdolecek remove all logic around ATACH_IRQ_WAIT and channel-global ch_error/ch_status,
so that there is less hidden state shared by commands; primary intent is
to make the NCQ and non-NCQ paths more similar, and remove possibility
of incorrect handling for the NCQ commands

tested both disk and ATAPI - piixide(4) on QEMU, and siisata(4),
ahcisata(4), mvsata(4) on real hw
 1.35.6.19  12-Aug-2017  jdolecek fix condition for switching to kernel thread for ATAPI and bio reset recovery
to match atapi_wdc.c, __wdcwait() postpones to the thread whenever neither
AT_POLL nor AT_WAIT flags are present

fixes spurious 'timeout' for the command on mvsata, followed by 'unexpected'
interrupt
 1.35.6.18  28-Jun-2017  jdolecek enable ATAPI on mvsata(4); it seems to work fine for my cdrom
 1.35.6.17  27-Jun-2017  jdolecek attend error paths, more strict asserts and code consistency

- atastart() and ata_kill_pending() now KASSERT() that all xfers on queue
have same channel
- inactive xfers are killed via new reason KILL_GONE_INACTIVE, controller
code must not call any resource deactivation in that case
- c_intr() must call ata_waitdrain_xfer_check() as first thing, and must not
further touch any xfer structures on exit path; any resource cleanup
is supposed to be done in c_kill_xfer()
- c_kill_xfer() should never call atastart()
- ata_waitdrain_check() removed, replaced by ata_waitdrain_xfer_check()
- ATA_DRIVE_WAITDRAIN handling converted to use condvar
- removed unused ata_c callback
 1.35.6.16  24-Jun-2017  jdolecek fix confusion around AT_WAIT vs AT_POLL for mvsata_edma_disable(), and switch
all the tsleep()/delay() ifs to just use ata_delay()
 1.35.6.15  24-Jun-2017  jdolecek fix dump on mvsata - there was misplaced mvsata_bio_intr() call causing
diagnostic panic, and we also need to switch controller to appropriate
DMA mode (NCQ vs. non-NCQ DMA)

while here, also reduce delay for the polled command wait to 100 us, same
as siisata(4) and ahcisata(4)
 1.35.6.14  21-Jun-2017  jdolecek change ata_queue_hwslot_to_xfer() and ata_queue_get_active_xfer() to take
ata_channel instead of ata_queue as parameter, and lock the channel while
traversing the queue
 1.35.6.13  20-Jun-2017  jdolecek remove ata queue downsizing - every device, attached to the same channel,
uses slots according to it's own limits

wdc code changed to expect maximum one active xfer, and not check number
of openings in the channel; this is to facilitate using wdc functions
for e.g. handling of atapi commands for drivers which support both ATAPI
and NCQ
 1.35.6.12  19-Jun-2017  jdolecek add ata_channel lock, use it to protect queue manipulation (only that for now);
add ata_channel_detach() to destroy the locks

change ata_get_xfer() so that it can wait for xfer, convert all on-stack
xfer code to use the blocking variant

fix siisata_reset_drive() to use polled reset and not try ata_activate_xfer(),
convert drive probe code also over from slot0 XXX to ata_get_xfer()

drive reset and PMP now works on siisata(4) too; changes tested also
on piixide(4), ahci(4), mvsata(4)
 1.35.6.11  16-Jun-2017  jdolecek adjust reset channel and dump paths
- channel reset now always kills active transfer, even on dump path, but
now doesn't touch the queued waiting transfers; also kill_xfer hook is
always called, so that HBA can free any private xfer resources and thus
the dump request has chance to work
- kill_xfer routines now always call ata_deactivate_xfer(); added KASSERT()s
to ata_free_xfer() to expect deactivated xfer
- when called during channel reset before dump, ata_kill_active() drops
any queued waiting transfers without processing
- do not (re)queue any transfers in wddone() when dumping
- kill AT_RST_NOCMD flag

This should also hopefully fix the 'polled command has been queued' panic
as reported in:
PR kern/11811 by John Hawkinson
PR kern/47041 by Taylor R Campbell
PR kern/51979 by Martin Husemann

dump tested working with piixide(4) and ahci(4). mvsata(4) dump times out,
but otherwise tested working, will be fixed separately. siisata(4) mechanically
changed and not tested.
 1.35.6.10  13-Jun-2017  jdolecek use MVSATA_EDMAQ_LEN instead of magic number for number of queue slots
 1.35.6.9  12-Jun-2017  jdolecek add support for NCQ on Gen IIe hardware

mvsata_quetag_get()/put() pairs adjusted to work also for non-DMA code paths,
i.e. during probe; replaced wdcintr() call with local routine which uses
currently active command tag, rather than assuming always tag 0 on that
code path

~55MB/s -> ~80MB/s sequential read using fio(1) on a test HDD, quite nice
 1.35.6.8  10-Jun-2017  jdolecek use satafis routine to build the command block for Gen IIe to reduce code
duplication
 1.35.6.7  09-Jun-2017  jdolecek fix the driver to work again - switch to using tag from xfer, adjust
to framework changes (queue dynamically allocated), and do not store
pointer to xfer inside internal structures

unhide the ATAPI code and fix to compile, keep it disabled however

no indended functional changes, logic kept as close as possible to what was
there before; tested with Adaptec 1430SA (88SX7042)
 1.35.6.6  19-Apr-2017  jdolecek adjust ata code to support more than one active command, including the
timeout handling, add support for NCQ commands

move probe for NCQ and number of tags to middle layer, negotiate mutual
support between drive and controller

implement NCQ support in ahci(4)
 1.35.6.5  15-Apr-2017  jdolecek pass also ata_command via ata_xfer, callers of ata_exec_command() is now
responsible for allocation/disposal of the structure

change code to allocate ata_xfer for commands on stack same way as previously
the ata_command were, using c_slot 0; adjust asserts so that it would allow
several xfers with same c_slot, as long as only one such transfer is active
at a time
 1.35.6.4  15-Apr-2017  jdolecek make ata_xfer's allocated as part of ata_queue and make it include ata_bio;
they are pre-allocated on attach and ata_get_xfer() now never sleep, drop the
pool

modify wd(4) to file the bio requests using the xfers and hence
make it possible to have more than one active I/O request in flight;
ata_bio callback doesn't need to allocate any memory any more,
require it to never return ATACMD_TRY_AGAIN

move lp, badsect, multi from ata_bio to ata_drive_datas, as they are per-drive,
not per transfer

drop unused drv atac_claim_hw/atac_free_hw hooks, and also drop again ata_bio
c_hwslot
 1.35.6.3  11-Apr-2017  jdolecek remove left-over curly brace
 1.35.6.2  11-Apr-2017  jdolecek fix to not access active_xfer directly
 1.35.6.1  10-Apr-2017  jdolecek ATA infrastructure improvements to eventually support more outstanding
commands

patch by Matt Thomas
 1.39.2.3  26-Nov-2018  pgoyette Sync with HEAD, resolve a couple of conflicts
 1.39.2.2  06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.39.2.1  16-Apr-2018  pgoyette Sync with HEAD, resolve some conflicts
 1.40.2.4  21-Apr-2020  martin Sync with HEAD
 1.40.2.3  13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.40.2.2  08-Apr-2020  martin Merge changes from current as of 20200406
 1.40.2.1  10-Jun-2019  christos Sync with HEAD
 1.41.2.11  15-Oct-2018  jdolecek change the SATA/NCQ recovery to run in the atabus thread
 1.41.2.10  14-Oct-2018  jdolecek adjust mvsata_bio_intr() so it recognizes the 'tfd' parameter as passed
by recovery and hence works; use it also for passing state from
mvsata_edma_handle()
 1.41.2.9  13-Oct-2018  jdolecek make compile again with MVSATA_WITHOUTDMA
 1.41.2.8  13-Oct-2018  jdolecek add C_TIMEOUT KASSERT() on fail path of mvsata_bio_poll() for clarity
 1.41.2.7  11-Oct-2018  jdolecek refactor shared parts of the SATA error recovery into new function
ata_recovery_resume() and use for ahcisata/siisata/mvsata, also replace
per-controller hold/unhold with generic version

move the shared recovery code into separate file ata_recovery.c
 1.41.2.6  04-Oct-2018  jdolecek further channel locking pass for reset
 1.41.2.5  03-Oct-2018  jdolecek change channel reset and drive reset for all ATA controllers to always
run via thread, and with channel lock held the whole time; the queue is
frozen while reset is pending

for this repurpose ata_reset_channel() into new ata_thread_run()

also adjust some device printfs to not leak xfer pointer, and avoid
aprint_* for non-autoconf messages
 1.41.2.4  22-Sep-2018  jdolecek separate ata_xfer slot allocation and the memory allocation, so that
there can be more queued xfers than number of supported slots by controller,
and use a pool instead of custom pre-allocation

primarily to help PR kern/52614

remove no longer needed custom wd(4) logic for flush cache

switch also wd(4) trim/suspend/setcache/wdioctlstrategy to sleep waiting
for the memory, they are all called from process context and this
avoids spurious failures
 1.41.2.3  17-Sep-2018  jdolecek move ATAPI-only members of ata_xfer to an union struct to further save space
 1.41.2.2  17-Sep-2018  jdolecek move low-level protocol handlers hooks from ata_xfer to separate struct,
initialized statically

primarily to reduce ata_xfer struct size, but also improves readibility,
and enforces consistency
 1.41.2.1  31-Aug-2018  jdolecek refactor ata_xfer to be just dumb structure; move all callouts/condvars out

retry callout to wd(4); reset callout and the active/cmd finish condvars
to channel queue; change code using the condvars so it works if there
are multiple waiters

simplify the async wait code for cmds, replace ata_wait_xfer()/ata_wake_xfer()
with ata_wait_cmd()

fix the callout_invoking/ack race handling code for timeouts to
actually have chance to work; change mvsata(4) to use generic timeout func

towards resolution of kern/52614
 1.48.2.1  30-Dec-2022  martin Pull up following revision(s) (requested by tsutsui in ticket #1557):

sys/dev/ic/ahcisata_core.c: revision 1.83
sys/dev/ic/ahcisata_core.c: revision 1.102
sys/dev/ata/ata.c: revision 1.164
sys/dev/ata/ata_wdc.c: revision 1.115
sys/dev/ata/ata_recovery.c: revision 1.4
sys/dev/ic/siisata.c: revision 1.42
sys/dev/ic/wdc.c: revision 1.308
sys/dev/ic/mvsata.c: revision 1.56
sys/dev/scsipi/atapi_wdc.c: revision 1.138
sys/dev/ic/siisata.c: revision 1.49
sys/dev/ata/atavar.h: revision 1.105
sys/dev/ata/wd.c: revision 1.460
sys/dev/ata/ata.c: revision 1.155
sys/dev/ata/wd.c: revision 1.462
sys/dev/ata/atavar.h: revision 1.109
sys/dev/ata/satapmp_subr.c: revision 1.16
sys/dev/ic/wdc.c: revision 1.299
sys/dev/ic/ahcisata_core.c: revision 1.93
sys/dev/ata/ata_wdc.c: revision 1.120
sys/dev/ic/wdcvar.h: revision 1.100
sys/dev/scsipi/atapi_wdc.c: revision 1.141
sys/dev/ic/mvsata.c: revision 1.61
sys/dev/usb/umass_isdata.c (apply patch)

drop wd lock in wdstart1() before calling the ata_bio hook; when called
from ata thread context, that can still need to sleep for wdc attachments
in wdcwait()

fix use-after-free for ata xfer on bio submission found by KASAN
driver ata_bio hooks read parts of the xfer after ata_exec_xfer()
call in order to determine return value, change so that the hook
doesn't return any value - callers do not care already,
as all I/O requests are asynchronous

this problem was uncovered by recent change for wd(4) to not hold
wd mutex during ata_bio call, the interrupt for the xfer might
thus actually fire immediately

adjust also ata_exec_command driver hooks similarily - remove all
completion and waiting logic from drivers, upper layer ata code
using AT_WAIT/AT_POLL changed to call ata_wait_cmd() itself
PR kern/55169 by Nick Hudson

Function declaration formating whitespace consistency. NFCI.

PR kern/56403
Fix kernel freeze for wdc(4) variants with ATAC_CAP_NOIRQ:

(1) Change ata_xfer_ops:c_poll from void to int function. When it returns
ATAPOLL_AGAIN, let ata_xfer_start() iterate itself again.
(2) Let wdc_ata_bio_poll() return ATAPOLL_AGAIN until ATA_ITSDONE is
achieved.

A similar change has been made for mvsata(4) (see mvsata_bio_poll()),
and no functional changes for other devices.

This is how the drivers worked before jdolecek-ncq branch was merged.
Note that this changes are less likely to cause infinite recursion:

(1) wdc_ata_bio_intr() called from wdc_ata_bio_poll() asserts ATA_ITSDONE
in its error handling paths via wdc_ata_bio_done().
(2) Return value from c_start (= wdc_ata_bio_start()) is checked in
ata_xfer_start().

Therefore, errors encountered in ata_xfer_ops:c_poll and c_start routines
terminate the recursion for wdc(4). The situation is similar for mvsata(4).

Still, there is a possibility where ata_xfer_start() takes long time to
finish a normal operation. This can result in a delayed response for lower
priority interrupts. But, I've never observed such a situation, even when
heavy thrashing takes place for swap partition in wd(4).
"Go ahead" by jdolecek@.
 1.52.2.1  29-Feb-2020  ad Sync with head.
 1.55.2.1  20-Apr-2020  bouyer Sync with HEAD
 1.57.2.1  03-Jan-2021  thorpej Sync w/ HEAD.
 1.58.2.1  02-Apr-2021  thorpej config_found_ia() -> config_found() w/ CFARG_IATTR.
 1.59.8.1  04-Aug-2021  thorpej Adapt to CFARGS().

RSS XML Feed