Home | History | Annotate | Download | only in ic
History log of /src/sys/dev/ic/nvme.c
RevisionDateAuthorComments
 1.69  11-Mar-2024  riastradh nvme(4): Disestablish admin q interrupt while suspended.

And re-establish on resume. Fixes nvmectl(8) after a suspend/resume
cycle on some systems.

Adapted from a patch by mrg@.

PR kern/58025
 1.68  10-Mar-2024  mrg nvme(4): don't write to INTMC register if in intx mode

this matches the handling of INTMS, both of which have the same
restrictions on access in MSI-X mode.

ideally, this register should be written to with a full set of
values for MSI (upto 32 bits), but trying to force MSI mode for
my test machine makes the system unstable.

tested with samsung SM981 256GB, samsung 980 PRO 1TB, and intel
760p 250G.
 1.67  13-Sep-2022  riastradh branches: 1.67.4;
nvme(4): Don't leak memory for queues on every resume.
 1.66  30-Aug-2022  riastradh nvme(4): Actually check if bp is null as commented previously.

I had tested this change, but forgot to amend the commit before
exporting to CVS.
 1.65  30-Aug-2022  riastradh nvme(4): If bp is null or bp->b_ci is not assigned, use curcpu().

curcpu() might be stale by the time we're done, but it's still safe
to pass it to cpu_index, and this is just used as a best-effort
mechanism to keep I/O on queues handled by the same CPU.

bp is not always provided, and bp->b_ci is not always assigned,
e.g. when dumping. (If bp->b_ci is supposed to be always assigned,
then we need to audit all the paths into it to assign it in those
where it's not.)

Fixes dump on nvme.
 1.64  20-Aug-2022  riastradh nvme(4): Read cqe flags and cid in that order.
 1.63  15-Aug-2022  riastradh nvme(4): KASSERT(A && B) -> KASSERT(A); KASSERT(B)
 1.62  14-Aug-2022  jmcneill nvme: Make sure that q_ccb_list is always accessed with the q lock held.
 1.61  31-Jul-2022  mlelstv The namespace id is a 32bit value, in particular the "all namespaces" value
for global commands is 0xffffffff. While the driver only supports 16bit
numbers (device minor & 0xffff), we need to use the full value for pass
through commands.

This fixes e.g. logpage requests on the controller level.
 1.60  07-May-2022  skrll Add support for Apple silicon NVME. Ported from OpenBSD.
 1.59  16-Nov-2021  skrll Trailing whitespace
 1.58  07-Aug-2021  thorpej Merge thorpej-cfargs2.
 1.57  29-May-2021  riastradh branches: 1.57.4;
nvme(4): Move disestablishment of admin q interrupt to nvme_detach.

Nothing re-established this after suspend/resume, so attempting
suspend/resume/suspend would crash, and presumably we would miss
interrupts after resume. This keeps the establish/disestablish more
symmetric in attach/detach.
 1.56  29-May-2021  riastradh nvme(4): Add suspend/resume, derived from OpenBSD.
 1.55  24-Apr-2021  thorpej branches: 1.55.2; 1.55.4;
Merge thorpej-cfargs branch:

Simplify and make extensible the config_search() / config_found() /
config_attach() interfaces: rather than having different variants for
which arguments you want pass along, just have a single call that
takes a variadic list of tag-value arguments.

Adjust all call sites:
- Simplify wherever possible; don't pass along arguments that aren't
actually needed.
- Don't be explicit about what interface attribute is attaching if
the device only has one. (More simplification.)
- Add a config_probe() function to be used in indirect configuiration
situations, making is visibly easier to see when indirect config is
in play, and allowing for future change in semantics. (As of now,
this is just a wrapper around config_match(), but that is an
implementation detail.)

Remove unnecessary or redundant interface attributes where they're not
needed.

There are currently 5 "cfargs" defined:
- CFARG_SUBMATCH (submatch function for direct config)
- CFARG_SEARCH (search function for indirect config)
- CFARG_IATTR (interface attribte)
- CFARG_LOCATORS (locators array)
- CFARG_DEVHANDLE (devhandle_t - wraps OFW, ACPI, etc. handles)

...and a sentinel value CFARG_EOL.

Add some extra sanity checking to ensure that interface attributes
aren't ambiguous.

Use CFARG_DEVHANDLE in MI FDT, OFW, and ACPI code, and macppc and shark
ports to associate those device handles with device_t instance. This
will trickle trough to more places over time (need back-end for pre-OFW
Sun OBP; any others?).
 1.54  27-Dec-2020  jmcneill branches: 1.54.2;
Zero DMA memory after load, and add PREREAD sync op after to ensure it is visible
 1.53  04-Dec-2020  kardel PR kern/55839:
handle multiple nvme_rescan()s correctly by doing the
name-space identify only once per nsid.

fixes issue where modloading triggers multiple
rescans.
 1.52  10-Nov-2020  rin One more fix for big-endian machines.

Now, ld(4) at nvme(4) works perfectly for aarch64eb!

Tested on ROCKPro64 and qemu-system-aarch64.
 1.51  24-Sep-2020  ryo branches: 1.51.2;
no need to swap pt.cmd.cdwXX at nvme_set_number_of_queues(). cmd.cdwXX will be swapped in nvme_pt_fill().
 1.50  22-Sep-2020  kardel PR kern/55674:
move name space availability check from ld_nvme.c:ld_nvme_attach()
to nvme.c:nvme_rescan().
this avoids allocation of ld(4) instances for every possible
name space, even if it is not usable. it also reduces the device
node flood generated from that strategy.
 1.49  28-Jul-2020  jdolecek say what is not configured in nvme_print()
 1.48  07-Apr-2020  ryo avoid "panic: LOCKDEBUG: Mutex error: rw_vector_enter,309: spin lock held"

ok nonaka@. thanks
 1.47  11-Nov-2019  nonaka nvme(4): Use the SET_FEATURES command to get the number of allocated queues.
 1.46  26-Sep-2019  nonaka nvme(4): Don't attach the device, if namespace not found.
 1.45  20-Sep-2019  nonaka Don't set Phase Tag bit of Completion Queue entry at nvme_poll_done().

A new completion queue entry check incorrectly determined that there was
a Completion Queue entry for a command that was not submitted.

Fix PR kern/54275, PR kern/54503, PR kern/54532.
 1.44  28-Jun-2019  jmcneill branches: 1.44.2;
Fix a performance issue where one busy queue can starve all other queues.

In normal operations with multiple queues, the nvme driver will attempt
to schedule I/O requests on the submitting CPU. This breaks down when any
one of the queues becomes full; the driver returns EAGAIN to the disk
layer, which causes the disk layer to stop submitting more requests until
the blocked request is consumed. When space becomes available in the full
queue, it pulls the next buffer from the bufq and fills the queue again,
until finally hitting EAGAIN and preventing other queues from processing
requests.

Two changes here to fix the problem:

- When processing requests from the bufq, attempt to assign them to the
queue associated with the CPU that originated the request.
- If that queue is busy, try to find another queue with available space
before returning EAGAIN. This way, only when all queues are full will
the disk layer stop submitting more requests.

Now for some real numbers. On a Rockchip RK3399 board (6 CPUs), with 6
concurrent readers:

Old code:
4294967296 bytes transferred in 52.420 secs (81933752 bytes/sec)
4294967296 bytes transferred in 53.969 secs (79582117 bytes/sec)
4294967296 bytes transferred in 55.391 secs (77539082 bytes/sec)
4294967296 bytes transferred in 55.649 secs (77179595 bytes/sec)
4294967296 bytes transferred in 56.102 secs (76556402 bytes/sec)
4294967296 bytes transferred in 72.901 secs (58915066 bytes/sec)

New code:
4294967296 bytes transferred in 37.171 secs (115546186 bytes/sec)
4294967296 bytes transferred in 37.611 secs (114194445 bytes/sec)
4294967296 bytes transferred in 37.655 secs (114061009 bytes/sec)
4294967296 bytes transferred in 38.247 secs (112295534 bytes/sec)
4294967296 bytes transferred in 38.496 secs (111569183 bytes/sec)
4294967296 bytes transferred in 38.595 secs (111282997 bytes/sec)
 1.43  14-Jun-2019  mrg in nvme_attach() when creating the admin queue to probe the device info,
and also in nvme_dmamem_alloc(), allow as many DMA segment as would be
maximally needed for the size, rather than hard coding '2' for the form
and '1' for the latter.

now ld@nvme on i386 doesn't crash and i see at least 1.3GB/sec.
 1.42  24-Apr-2019  mlelstv Expose device type. You can query it with e.g. drvctl -p ld0 disk-info/type.
 1.41  01-Dec-2018  jdolecek support DIOCSCACHE + DKCACHE_WRITE if volatile write cache is present

fix the Get Features call for DIOCGCACHE to actually retrieve the current
value properly
 1.40  01-Dec-2018  jdolecek disestablish the interrupt on failure in nvme_q_create()
 1.39  18-Apr-2018  nonaka branches: 1.39.2;
nvmectl(8): Add big-endian support.

from FreeBSD nvmecontolr(8) r329824.
 1.38  18-Apr-2018  nonaka nvme(4): Added some delay before check RDY bit quirk when disabling device.

Pick from FreeBSD nvme(4) r326937.
 1.37  17-Mar-2018  jdolecek also remove now duplicate nvme_ccb_put() call from nvme_get_number_of_queues()
 1.36  17-Mar-2018  jdolecek fix passthrough command usage also in nvme_get_number_of_queues(), fixes
memory corruption and possible panic on boot

PR kern/53059
 1.35  17-Mar-2018  jdolecek switch handling of passthrough commands to use queue, instead of polling

should fix PR kern/53059 by Frank Kardel
 1.34  16-Mar-2018  jdolecek refactor the locking code around DIOCGCACHE handling to be reusable
for other infrequent commands

it uses single condvar for simplicity, and uses it both when waiting
for ccb or command completion - this is fine, since usually there
will be just one such command qeueued anyway

use this to finally properly implement DIOCCACHESYNC - return only after
the command is confirmed as completed by the controller
 1.33  16-Mar-2018  jdolecek stop using q_nccbs_avail for deciding whether there are available ccbs;
no need to maintain a counter _and_ q_ccb_list

this fixes deadlock when all ccbs happen to be taken before completion
interrupt - nvme_q_complete() increased q_nccbs_avail only after
processing all the completed commands, by then there was nothing
left to actually kick the disk queue again into action

this also fixes ccb leak on command errors e.g. with bus_dmamem_alloc()
or bus_dmamel_load() - q_nccbs_avail was never decreased on the error path

fixes PR kern/52769 by Martin Husemann, thanks to Paul Goyette
for testing
 1.32  27-Feb-2018  christos branches: 1.32.2;
- don't leak ccb on alloc failure.
- KASSERT to prevent memory leak.
 1.31  28-Oct-2017  riastradh branches: 1.31.2;
Kill some more extern struct cfdriver declarations.

Down with externs in .c!
 1.30  01-Jun-2017  chs branches: 1.30.2;
remove checks for failure after memory allocation calls that cannot fail:

kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()

all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.
 1.29  29-May-2017  nonaka nvme(4): Don't set prp1 for DEL_IOCQ

> NVM_ADMIN_DEL_IOCQ does not need prp1 (just as NVM_ADMIN_DEL_IOSQ).
> Remove what is likely a cut'n'paste error from the *_ADD_* code.

from OpenBSD nvme.c r1.56.
 1.28  29-May-2017  nonaka nvme(4): Do not use bus_space_{read,write}_8() even on LP64 archs.

> some (broken) controllers require ordered split transfers.
> From linux a310acd7a7ea53533886c11bb7edd11ffd61a036

from OpenBSD nvme.c r1.53.
 1.27  29-May-2017  nonaka nvme(4): Mask non relevant bits when pritting version number.

from OpenBSD nvme.c r1.52.
 1.26  05-Apr-2017  jdolecek branches: 1.26.4;
expose disk device FUA/DPO support via DIOCGCACHE, and allow the flags
to be set for I/O; implement support in sd(4) and nvme(4)

discussed on tech-kern
 1.25  28-Feb-2017  jdolecek implement DIOCGCACHE
 1.24  13-Feb-2017  nonaka NVMe 1.2.1
 1.23  13-Feb-2017  nonaka nvme(4): Limit the number of queues to the number allocated in HW.
 1.22  01-Nov-2016  jdolecek branches: 1.22.2;
reduce admin queue size to save memory; it's only ever used during
attach/detach and for nvmectl(8), so there is actually no point having it big
 1.21  01-Nov-2016  jdolecek pass maxphys from device rather then assuming MAXPHYS; it's clipped in ld(4)
if bigger then MAXPHYS

multiply the queue size by number of queues for ld(4) sc_maxqueuecnt, so
that ld_diskstart() would try to use full capacity, instead of throttling
to one queue worth of commands
 1.20  01-Nov-2016  jdolecek tighter queue control - according to spec actual cap on number of commands
in flight is actually one less then queue size, head == tail means empty
queue
 1.19  20-Oct-2016  jdolecek revert change from rev. 1.12:
"""
slightly optimize memory access - change struct nvme_queue so that the
struct dmamem members are allocated as part of it, instead of separate
kmem_alloc()s
"""

that change quite curiously caused completion queue corruption on MP systems,
regardless of MPSAFE setting for the pci/softintr interrupt
 1.18  19-Oct-2016  jdolecek add debug code to check for completion queue corruption
 1.17  19-Oct-2016  jdolecek follow advice of spec and block interrupts via INTMS/INTMC for intx handler;
this also makes it possible to offload the actual interrupt processing to softintr
handler, similar as for MSI/MSI-X
 1.16  18-Oct-2016  nonaka fix a sync range size when number of bus_dma segment is greater than 2.
 1.15  05-Oct-2016  nonaka The maximum number of IO queue entries is hardware supported value.
 1.14  27-Sep-2016  pgoyette Modularize the ld driver and all of its attachments. Ensure that all
parents are capable of rescan (or otherwise provide a means of attaching
children post-initialization).
 1.13  20-Sep-2016  jdolecek cache flush call must wait for completion; cap wait to 60 seconds just in case
controller won't respond
 1.12  19-Sep-2016  jdolecek slightly optimize memory access - change struct nvme_queue so that the
struct dmamem members are allocated as part of it, instead of separate
kmem_alloc()s
 1.11  19-Sep-2016  jdolecek on further thought, just remove the separately allocated nvme_ns_context
altogether and fold into nvme_ccb; allocating this separately just isn't useful
 1.10  19-Sep-2016  jdolecek comment tyop and KNF; pointed out by Nick
 1.9  18-Sep-2016  jdolecek fix several bugs, make nvme(4) MPSAFE by default and also bump default
number of ioq from 128 to 1024; tested with VirtualBox and QEMU

* remove NVME_INTMC/NVME_INTMS writes in hw intr handler as this is not MPSAFE,
fortunately they don't seem to be necessary; shaves two register writes
* need to use full mutex_enter() in nvme_q_complete(), to avoid small
race between one handler exiting the loop and another entering
* for MSI, handover the command result processing to softintr; unfortunately
can't easily do that for INTx interrupts as they require doorbell write
to deassert
* unlock/relock q->q_cq_mtx before calling ccb_done to avoid potential deadlocks
* make sure to destroy queue mutexes when destroying the queue (LOCKDEBUG)
* make ns ctx pool per-device, so that it's deallocated properly on module
unload
* handle ctx allocation failure in ld_nvme_dobio()
* remove splbio() calls in ld_nvme_dobio() and sync, the paths are exercised
only for dump/shutdown, and that already disables interrupts
* free the ns ctx in ld_nvme_biodone() before calling lddone() to avoid
memory starvation, as lddone() can trigger another i/o request
* be more careful with using PR_WAITOK, the paths are called from interrupt
context and there we can't wait
 1.8  17-Sep-2016  jdolecek make nvme_dumpregs() compile on LP64 hosts

sprinkle some delays() and extra checks into attach code, so that it follows
more closely what FreeBSD driver does, and is easier to cross-check
 1.7  16-Sep-2016  jdolecek introduce optional timeout for the polled commands; currently 5 seconds for
queue creation/deletion, 10 seconds for controller/namespace identify,
and unlimited for cache sync and passthrough commands

this makes device attach error out instead of hanging the kernel when
the device fails to respond properly, such as under QEMU currently
 1.6  16-Sep-2016  jdolecek make nvme_dumpregs() compile when NVME_DEBUG is defined
 1.5  08-Sep-2016  nonaka Fix to cannnot be opened a nvme(4) namespace device file.
 1.4  08-Sep-2016  nonaka fix off-by-one error in namespace id range check.
 1.3  04-Jun-2016  nonaka branches: 1.3.2;
Add NVMe command passthrough support.
 1.2  02-May-2016  christos branches: 1.2.2;
move scsipi_strvis -> libkern:strnvisx()
change the prototype to match userland
fix sizes of strings passed to it
 1.1  01-May-2016  nonaka Added nvme(4) for Non-Volatile Memory Host Controller Interface devices.
Ported from OpenBSD.
 1.2.2.6  28-Aug-2017  skrll Sync with HEAD
 1.2.2.5  05-Dec-2016  skrll Sync with HEAD
 1.2.2.4  05-Oct-2016  skrll Sync with HEAD
 1.2.2.3  09-Jul-2016  skrll Sync with HEAD
 1.2.2.2  29-May-2016  skrll Sync with HEAD
 1.2.2.1  02-May-2016  skrll file nvme.c was added on branch nick-nhusb on 2016-05-29 08:44:21 +0000
 1.3.2.3  26-Apr-2017  pgoyette Sync with HEAD
 1.3.2.2  20-Mar-2017  pgoyette Sync with HEAD
 1.3.2.1  04-Nov-2016  pgoyette Sync with HEAD
 1.22.2.1  21-Apr-2017  bouyer Sync with HEAD
 1.26.4.1  30-Apr-2017  pgoyette And a few more DEVSW_MODULE_INIT for modular drivers
 1.30.2.9  07-Dec-2020  martin Pull up following revision(s) (requested by kardel in ticket #1633):

sys/dev/ic/nvme.c: revision 1.53

PR kern/55839:

handle multiple nvme_rescan()s correctly by doing the
name-space identify only once per nsid.
fixes issue where modloading triggers multiple
rescans.
 1.30.2.8  27-Sep-2020  martin Pull up following revision(s) (requested by kardel in ticket #1610):

sys/dev/ic/ld_nvme.c: revision 1.24 (patch)
sys/dev/ic/nvme.c: revision 1.50 (patch)

PR kern/55674:
move name space availability check from ld_nvme.c:ld_nvme_attach()
to nvme.c:nvme_rescan().
this avoids allocation of ld(4) instances for every possible
name space, even if it is not usable. it also reduces the device
node flood generated from that strategy.
 1.30.2.7  11-Nov-2019  martin Pull up following revision(s) (requested by nonaka in ticket #1432):

sys/dev/ic/nvme.c: revision 1.47

nvme(4): Use the SET_FEATURES command to get the number of allocated queues.
 1.30.2.6  26-Sep-2019  martin Pull up following revision(s) (requested by nonaka in ticket #1390):

sys/dev/ic/nvme.c: revision 1.46

nvme(4): Don't attach the device, if namespace not found.
 1.30.2.5  25-Sep-2019  martin Pull up following revision(s) (requested by nonaka in ticket #1386):

sys/dev/ic/nvme.c: revision 1.45

Don't set Phase Tag bit of Completion Queue entry at nvme_poll_done().

A new completion queue entry check incorrectly determined that there was
a Completion Queue entry for a command that was not submitted.

Fix PR kern/54275, PR kern/54503, PR kern/54532.
 1.30.2.4  19-Apr-2018  martin Pull up following revision(s) (requested by nonaka in ticket #781):

sbin/nvmectl/Makefile 1.4
sbin/nvmectl/bignum.c 1.2
sbin/nvmectl/devlist.c 1.3-1.5
sbin/nvmectl/firmware.c 1.3,1.4
sbin/nvmectl/identify.c 1.3-1.5
sbin/nvmectl/logpage.c 1.5-1.7
sbin/nvmectl/nvme.h 1.3
sbin/nvmectl/nvmectl.8 1.5
sbin/nvmectl/nvmectl.c 1.5-1.7
sbin/nvmectl/nvmectl.h 1.5-1.8
sbin/nvmectl/perftest.c 1.3-1.5
sbin/nvmectl/power.c 1.3,1.4
sbin/nvmectl/reset.c 1.2,1.3
sbin/nvmectl/util.c 1.1,1.2
sbin/nvmectl/wdc.c 1.2-1.4
sys/dev/ic/ld_nvme.c 1.20
sys/dev/ic/nvme.c 1.38,1.39
sys/dev/ic/nvmeio.h 1.2
sys/dev/ic/nvmereg.h 1.10,1.11
sys/dev/ic/nvmevar.h 1.16
sys/dev/pci/nvme_pci.c 1.20

nvmectl(8): Sync with FreeBSD nvmecontrol(8) r328763.

nvmectl(8): fix wdc command usage.

nvme(4): Added some delay before check RDY bit quirk when disabling device.
Pick from FreeBSD nvme(4) r326937.

Add some new structure fileds, opcodes and statuses from NVMe 1.3a.

nvmectl(8): Add big-endian support.
from FreeBSD nvmecontolr(8) r329824.

nvmectl(8): fix subcommand usage.

nvmectl(8): Remove some wdc subcommands from man page.
- wdc drive-log
- wdc get-crash-dump
- wdc purge
- wdc purge-monitor

Typos.

use setprogname()/getprogname(), do not hardcode the prognam name in fixed
strings
 1.30.2.3  09-Apr-2018  bouyer Pull up following revision(s) (requested by msaitoh in ticket #718):
sys/dev/ic/nvme.c: revision 1.32
- don't leak ccb on alloc failure.
- KASSERT to prevent memory leak.
 1.30.2.2  18-Mar-2018  martin Pull up following revision(s) (requested by jdolecek in ticket #641):
sys/dev/ic/nvme.c: revision 1.34
sys/dev/ic/nvme.c: revision 1.35
sys/dev/ic/nvme.c: revision 1.36
sys/dev/ic/nvme.c: revision 1.37
sys/dev/ic/ld_nvme.c: revision 1.19
sys/dev/ic/nvmevar.h: revision 1.15

refactor the locking code around DIOCGCACHE handling to be reusable
for other infrequent commands,it uses single condvar for simplicity,
and uses it both when waiting for ccb or command completion - this
is fine, since usually there will be just one such command qeueued anyway
use this to finally properly implement DIOCCACHESYNC - return only after
the command is confirmed as completed by the controller.

switch handling of passthrough commands to use queue, instead of polling
should fix PR kern/53059 by Frank Kardel

fix passthrough command usage also in nvme_get_number_of_queues(), fixes
memory corruption and possible panic on boot

also remove now duplicate nvme_ccb_put() call from
nvme_get_number_of_queues()
 1.30.2.1  17-Mar-2018  martin Pull up following revision(s) (requested by jdolecek in ticket #636):
sys/dev/ic/nvme.c: revision 1.33
sys/dev/ic/nvmevar.h: revision 1.14
stop using q_nccbs_avail for deciding whether there are available ccbs;
no need to maintain a counter _and_ q_ccb_list
this fixes deadlock when all ccbs happen to be taken before completion
interrupt - nvme_q_complete() increased q_nccbs_avail only after
processing all the completed commands, by then there was nothing
left to actually kick the disk queue again into action
this also fixes ccb leak on command errors e.g. with bus_dmamem_alloc()
or bus_dmamel_load() - q_nccbs_avail was never decreased on the error path
fixes PR kern/52769 by Martin Husemann, thanks to Paul Goyette
for testing
 1.31.2.2  03-Dec-2017  jdolecek update from HEAD
 1.31.2.1  28-Oct-2017  jdolecek file nvme.c was added on branch tls-maxphys on 2017-12-03 11:37:03 +0000
 1.32.2.3  26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.32.2.2  22-Apr-2018  pgoyette Sync with HEAD
 1.32.2.1  22-Mar-2018  pgoyette Synch with HEAD, resolve conflicts
 1.39.2.2  13-Apr-2020  martin Mostly merge changes from HEAD upto 20200411
 1.39.2.1  10-Jun-2019  christos Sync with HEAD
 1.44.2.7  12-Mar-2024  martin Pull up following revision(s) (requested by riastradh in ticket #1817):

sys/dev/ic/nvme.c: revision 1.69

nvme(4): Disestablish admin q interrupt while suspended.

And re-establish on resume. Fixes nvmectl(8) after a suspend/resume
cycle on some systems.

Adapted from a patch by mrg@.

PR kern/58025
 1.44.2.6  21-Jun-2021  martin Pull up following revision(s) (requested by riastradh in ticket #1305):

sys/dev/ic/nvmevar.h: revision 1.22
sys/dev/ic/nvme.c: revision 1.56
sys/dev/ic/nvme.c: revision 1.57
sys/dev/pci/nvme_pci.c: revision 1.30

nvme(4): Add suspend/resume, derived from OpenBSD.

nvme(4): Move disestablishment of admin q interrupt to nvme_detach.

Nothing re-established this after suspend/resume, so attempting
suspend/resume/suspend would crash, and presumably we would miss
interrupts after resume. This keeps the establish/disestablish more
symmetric in attach/detach.
 1.44.2.5  07-Dec-2020  martin Pull up following revision(s) (requested by kardel in ticket #1144):

sys/dev/ic/nvme.c: revision 1.53

PR kern/55839:

handle multiple nvme_rescan()s correctly by doing the
name-space identify only once per nsid.
fixes issue where modloading triggers multiple
rescans.
 1.44.2.4  27-Sep-2020  martin Pull up following revision(s) (requested by kardel in ticket #1094):

sys/dev/ic/ld_nvme.c: revision 1.24
sys/dev/ic/nvme.c: revision 1.50

PR kern/55674:
move name space availability check from ld_nvme.c:ld_nvme_attach()
to nvme.c:nvme_rescan().
this avoids allocation of ld(4) instances for every possible
name space, even if it is not usable. it also reduces the device
node flood generated from that strategy.
 1.44.2.3  11-Nov-2019  martin Pull up following revision(s) (requested by nonaka in ticket #415):

sys/dev/ic/nvme.c: revision 1.47

nvme(4): Use the SET_FEATURES command to get the number of allocated queues.
 1.44.2.2  26-Sep-2019  martin Pull up following revision(s) (requested by nonaka in ticket #248):

sys/dev/ic/nvme.c: revision 1.46

nvme(4): Don't attach the device, if namespace not found.
 1.44.2.1  22-Sep-2019  martin Pull up following revision(s) (requested by nonaka in ticket #218):

sys/dev/ic/nvme.c: revision 1.45

Don't set Phase Tag bit of Completion Queue entry at nvme_poll_done().

A new completion queue entry check incorrectly determined that there was
a Completion Queue entry for a command that was not submitted.

Fix PR kern/54275, PR kern/54503, PR kern/54532.
 1.51.2.2  03-Jan-2021  thorpej Sync w/ HEAD.
 1.51.2.1  14-Dec-2020  thorpej Sync w/ HEAD.
 1.54.2.2  28-Mar-2021  thorpej - In nvme_rescan(), pass the locators used for matching to config_found()
and use config_stdsubmatch().
- No need to be explicit about interface attribute, because we carry
only one.
 1.54.2.1  21-Mar-2021  thorpej Give config_found() the same variadic arguments treatment as
config_search(). This commit only adds the CFARG_EOL sentinel
to the existing config_found() calls. Conversion of config_found_sm_loc()
and config_found_ia() call sites will be in subsequent commits.
 1.55.4.1  31-May-2021  cjep sync with head
 1.55.2.1  17-Jun-2021  thorpej Sync w/ HEAD.
 1.57.4.1  04-Aug-2021  thorpej Adapt to CFARGS().
 1.67.4.1  12-Mar-2024  martin Pull up following revision(s) (requested by riastradh in ticket #633):

sys/dev/ic/nvme.c: revision 1.69

nvme(4): Disestablish admin q interrupt while suspended.

And re-establish on resume. Fixes nvmectl(8) after a suspend/resume
cycle on some systems.

Adapted from a patch by mrg@.

PR kern/58025

RSS XML Feed