Home | History | Annotate | Download | only in pci
History log of /src/sys/dev/pci/nvme_pci.c
RevisionDateAuthorComments
 1.37  15-Aug-2022  pgoyette Remove stray debug line (already commented out)
 1.36  09-Jul-2022  pgoyette Remove redundant ``devsw_ok = false;''

Thanks mrg@
 1.35  08-Jul-2022  pgoyette devsw_ok needs to survive across invocations of nvme_modcmd() so
allocate it statically.

Should address remaining issues with kern/56914
 1.34  08-Jul-2022  pgoyette Initialize cmajor & bmajor so the devsw_attach() has a chance of
succeeding.

Record the success of devsw_attach(), and do not try later to
devsw_detach() unless the attach succeeded.

Partial fix for kern/56914
 1.33  07-Jul-2022  pgoyette Add errno tto message when fail to attach the devsw
 1.32  31-Mar-2022  pgoyette For device modules that provide both auto-config and /dev/xxx
interfaces, make sure that initialization and destruction
follow the proper sequence. This is triggered by the recent
changes to the devsw stuff; per riastradh@ the required call
sequence is:

devsw_attach()
config_init_component() or config_cf*_attach()
...
config_fini_component() or config_cf*_detach()
devsw_detach()

While here, add a few missing calls to some of the detach
routines.

Testing of these changes has been limited to:
1. compile without build break
2. no related test failures from atf
3. modload/modunload work as well as
before.

No functional device testing done, since I don't have any
of these devices. Let me know of any damage I might cause
here!

XXX Some of the modules affected by this commit are already
XXX broken; see kern/56772. This commit does not break
any additional modules (as far as I know).
 1.31  10-Oct-2021  skrll Catch up with pcireg.h change
 1.30  29-May-2021  riastradh nvme(4): Add suspend/resume, derived from OpenBSD.
 1.29  29-Jul-2020  jdolecek branches: 1.29.6; 1.29.8;
rename label now that it has dual purpose; suggested by mrg@
 1.28  28-Jul-2020  jdolecek add a quirk to disable MSI, and enable it for Intel SSD DC P4500

this device seems to cause serious system responsiveness issues when configured
to use MSI, while it works fine when configured for either INTx or MSI-X

this is important so this works well under Xen Dom0, which doesn't
support MSI-X yet

fixes another issue reported as feedback for PR port-xen/55285 by Frank Kardel
 1.27  02-Dec-2019  msaitoh Use PCI_MSIX_"TBL"BIR_MASK instead of PCI_MSIX_"PBA"BIR_MASK for MSI-X table.
This is not a real bug because both macros have the same value.
 1.26  23-Jan-2019  msaitoh branches: 1.26.4;
Nowadays some UEFI BIOSes don't enable some PCI devices' address decoding.
To resolve this problem, pci_map.c rev. 1.34-1.36 changed the
pci_mapreg_(sub)map()'s to set the decode bit if it's not set. It's good for
almost all drivers, but some other drivers don't use pci_mapreg_map().
In drivers which don't use pci_mapreg_map(), some of them expilicitly enable
decoding but others don't. Add code to enable decoding to them.

See also the following discussion:
http://mail-index.netbsd.org/tech-kern/2017/03/22/msg021678.html
 1.25  07-Dec-2018  msaitoh No functional change. OK'd by nonaka:
- Use correct macro.
- Don't use magic number.
 1.24  01-Dec-2018  jdolecek fix pasto (MSI -> MSIX)
 1.23  01-Dec-2018  jdolecek allow MSI-X even with just one interrupt, on some systems MSI/INTx might
not be available (or working, for that matter)
 1.22  01-Dec-2018  jdolecek simplify the interrupt allocation - it's not necessary to do the explicit
fallbacks, pci_intr_alloc() does this already internally
 1.21  03-Sep-2018  riastradh Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)
 1.20  18-Apr-2018  nonaka branches: 1.20.2;
nvme(4): Added some delay before check RDY bit quirk when disabling device.

Pick from FreeBSD nvme(4) r326937.
 1.19  01-Jun-2017  chs branches: 1.19.2; 1.19.8; 1.19.10;
remove checks for failure after memory allocation calls that cannot fail:

kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()

all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.
 1.18  13-Feb-2017  nonaka remove extra crlf.
 1.17  25-Nov-2016  knakahara branches: 1.17.2;
"#ifdef __HAVE_PCI_MSI_MSIX" workaround is not required any more.

see sys/dev/pci/pci_stub.c r1.6
 1.16  19-Oct-2016  jdolecek follow advice of spec and block interrupts via INTMS/INTMC for intx handler;
this also makes it possible to offload the actual interrupt processing to softintr
handler, similar as for MSI/MSI-X
 1.15  27-Sep-2016  pgoyette Modularize the ld driver and all of its attachments. Ensure that all
parents are capable of rescan (or otherwise provide a means of attaching
children post-initialization).
 1.14  18-Sep-2016  jdolecek fix several bugs, make nvme(4) MPSAFE by default and also bump default
number of ioq from 128 to 1024; tested with VirtualBox and QEMU

* remove NVME_INTMC/NVME_INTMS writes in hw intr handler as this is not MPSAFE,
fortunately they don't seem to be necessary; shaves two register writes
* need to use full mutex_enter() in nvme_q_complete(), to avoid small
race between one handler exiting the loop and another entering
* for MSI, handover the command result processing to softintr; unfortunately
can't easily do that for INTx interrupts as they require doorbell write
to deassert
* unlock/relock q->q_cq_mtx before calling ccb_done to avoid potential deadlocks
* make sure to destroy queue mutexes when destroying the queue (LOCKDEBUG)
* make ns ctx pool per-device, so that it's deallocated properly on module
unload
* handle ctx allocation failure in ld_nvme_dobio()
* remove splbio() calls in ld_nvme_dobio() and sync, the paths are exercised
only for dump/shutdown, and that already disables interrupts
* free the ns ctx in ld_nvme_biodone() before calling lddone() to avoid
memory starvation, as lddone() can trigger another i/o request
* be more careful with using PR_WAITOK, the paths are called from interrupt
context and there we can't wait
 1.13  18-Sep-2016  jdolecek attach also the nvme_cdevsw when loaded as module, so /dev/nvme* works
 1.12  17-Sep-2016  jdolecek remove dev code included in previous commit by mistake
 1.11  17-Sep-2016  jdolecek #ifdef out MSI related code on platforms not supporting MSI, so that it's
possible to load the driver as module on those platforms (the weak symbols
are not found by module load, claiming they don't exist); this makes it possible
to load the driver on e.g. sparc64, which is supposed to be working
under OpenBSD

unfortunately QEMU sparc64 emulator starts causing data access errors
on first device register read in nvme_attach(), so can't confirm the driver
is actually working on sparc64; same happens in QEMU when booting OpenBSD
image, so it seems to be emulator bug
 1.10  17-Sep-2016  jdolecek actually detach the ld {b,c}devsw on unload, and fail load if it's already
present
 1.9  17-Sep-2016  jdolecek need to set PCI_COMMAND_MASTER_ENABLE flag; fixes the driver under QEMU
 1.8  17-Sep-2016  pgoyette Since the ld code included with the nvme module references stuff in
the dk_subr module, add a dependency.
 1.7  16-Sep-2016  pgoyette We need sys/module.h for built-in modules as well as for the .kmod so
#include it unconditionally.
 1.6  16-Sep-2016  jdolecek make it possible to load nvme(4) as module to ease testing; currently somewhat
non-optimal, since it includes the ld(4) code also and hence requires the
kernel config to have 'no ld'
 1.5  16-Sep-2016  jdolecek fix nvme_pci_detach() to not pci_intr_disestablish(); the interrupts
are disestablished and handler pointers are set to NULL already by
nvme_detach(), leading to panic here; this was happening when running
on system without MSI/MSI-X

change all the alloc/dealloc code for sc_ih to use psc->psc_nintrs, so
that we avoid some duplicated logic
 1.4  11-Jul-2016  knakahara branches: 1.4.2;
pci_intr_type() is required pci_chipset_tag_t argument by other than x86.

pointed out by nonaka@n.o.
 1.3  24-Jun-2016  nonaka fix free memory size.
 1.2  11-May-2016  nonaka branches: 1.2.2;
Don't establish interrupt in nvme_pci_attach() when multi queue isn't used.
 1.1  01-May-2016  nonaka Added nvme(4) for Non-Volatile Memory Host Controller Interface devices.
Ported from OpenBSD.
 1.2.2.6  28-Aug-2017  skrll Sync with HEAD
 1.2.2.5  05-Dec-2016  skrll Sync with HEAD
 1.2.2.4  05-Oct-2016  skrll Sync with HEAD
 1.2.2.3  09-Jul-2016  skrll Sync with HEAD
 1.2.2.2  29-May-2016  skrll Sync with HEAD
 1.2.2.1  11-May-2016  skrll file nvme_pci.c was added on branch nick-nhusb on 2016-05-29 08:44:22 +0000
 1.4.2.3  20-Mar-2017  pgoyette Sync with HEAD
 1.4.2.2  07-Jan-2017  pgoyette Sync with HEAD. (Note that most of these changes are simply $NetBSD$
tag issues.)
 1.4.2.1  04-Nov-2016  pgoyette Sync with HEAD
 1.17.2.1  21-Apr-2017  bouyer Sync with HEAD
 1.19.10.4  26-Jan-2019  pgoyette Sync with HEAD
 1.19.10.3  26-Dec-2018  pgoyette Sync with HEAD, resolve a few conflicts
 1.19.10.2  06-Sep-2018  pgoyette Sync with HEAD

Resolve a couple of conflicts (result of the uimin/uimax changes)
 1.19.10.1  22-Apr-2018  pgoyette Sync with HEAD
 1.19.8.2  03-Dec-2017  jdolecek update from HEAD
 1.19.8.1  01-Jun-2017  jdolecek file nvme_pci.c was added on branch tls-maxphys on 2017-12-03 11:37:08 +0000
 1.19.2.3  03-Dec-2021  martin Pull up the following revisions, requested by msaitoh in ticket #1714:

sys/dev/pci/pcireg.h 1.148-1.154, 1.156-1.161
sys/dev/pci/pci_subr.c 1.217-1.222, 1.224, 1.227-1.232
via patch
sys/dev/pci/nvme_pci.c 1.31
sys/dev/pci/pci.c 1.158
sys/dev/pci/ppb.c 1.74

- Print Bridge Config Retry Enable bit and Retimer Presence Detect
Supported bit.
- Add PCIe 4.0 stuff a little:
- 10-bit Tag Requester/Completer.
- Add Data link Feature extended capability.
- Add Physical Layer 16.0 GT/s extended capability. Not decode yet.
- Change pci_conf_print() to allocate memory for the regs dynamically
instead of on-stack.
- Print some DPC register values not with %04x but with %08x because
those are 32bit.
- Fix a bug that the virtual channel extended configuration's
arbitration phase register can't be decoded correctly.
- When parsing Enhanced Allocation entries, use the correct calculation
for finding the next entry.
- Add 32.0GT/s to the list of pcie speeds (PCIe 5.x.).
- Add Some PCI config information:
- Lane Margining at the Receiver
- NVME admin interface
- UFSHCI
- InfiniBand
- Host fabric
- HDA 1.0 with vendor ext
- USB4 HCI
- MIPI I3C
- Cellular controller/modem (+ Ethernet)
- Change PCI_VENDOR_MASK and PCI_PRODUCT_MASK to unsigned values, to
prevent sign extension of product ID when shifted up into place in
PCI_ID_CODE(). Fixes PR kern/56176.
- Add LCAP & LCAP2 definitions.
- Use PCI-SIG official acronyms for some macros.
- Remove unused shift and mask definitions.
- Fix typo in some messages.
- Fix typo in comments.
- Whitespace fixes.
 1.19.2.2  27-Jan-2019  martin Pull up following revision(s) (requested by msaitoh in ticket #1172):

sys/dev/pci/nvme_pci.c: revision 1.26
sys/dev/pci/xhci_pci.c: revision 1.21
sys/dev/pci/ixgbe/ixv.c: revision 1.108
sys/dev/pci/ixgbe/ixgbe.c: revision 1.171
sys/dev/pci/if_fxp_pci.c: revision 1.84
sys/dev/pci/if_fxp_pci.c: revision 1.85
sys/dev/pci/xhci_pci.c: revision 1.16

remove #ifdef DEBUG printf, it seems to have outlived it's usefulness

-

KNF. No functional change.

-

Nowadays some UEFI BIOSes don't enable some PCI devices' address decoding.
To resolve this problem, pci_map.c rev. 1.34-1.36 changed the
pci_mapreg_(sub)map()'s to set the decode bit if it's not set. It's good for
almost all drivers, but some other drivers don't use pci_mapreg_map().
In drivers which don't use pci_mapreg_map(), some of them explicitly enable
decoding but others don't. Add code to enable decoding to them.

See also the following discussion:
http://mail-index.netbsd.org/tech-kern/2017/03/22/msg021678.html
 1.19.2.1  19-Apr-2018  martin Pull up following revision(s) (requested by nonaka in ticket #781):

sbin/nvmectl/Makefile 1.4
sbin/nvmectl/bignum.c 1.2
sbin/nvmectl/devlist.c 1.3-1.5
sbin/nvmectl/firmware.c 1.3,1.4
sbin/nvmectl/identify.c 1.3-1.5
sbin/nvmectl/logpage.c 1.5-1.7
sbin/nvmectl/nvme.h 1.3
sbin/nvmectl/nvmectl.8 1.5
sbin/nvmectl/nvmectl.c 1.5-1.7
sbin/nvmectl/nvmectl.h 1.5-1.8
sbin/nvmectl/perftest.c 1.3-1.5
sbin/nvmectl/power.c 1.3,1.4
sbin/nvmectl/reset.c 1.2,1.3
sbin/nvmectl/util.c 1.1,1.2
sbin/nvmectl/wdc.c 1.2-1.4
sys/dev/ic/ld_nvme.c 1.20
sys/dev/ic/nvme.c 1.38,1.39
sys/dev/ic/nvmeio.h 1.2
sys/dev/ic/nvmereg.h 1.10,1.11
sys/dev/ic/nvmevar.h 1.16
sys/dev/pci/nvme_pci.c 1.20

nvmectl(8): Sync with FreeBSD nvmecontrol(8) r328763.

nvmectl(8): fix wdc command usage.

nvme(4): Added some delay before check RDY bit quirk when disabling device.
Pick from FreeBSD nvme(4) r326937.

Add some new structure fileds, opcodes and statuses from NVMe 1.3a.

nvmectl(8): Add big-endian support.
from FreeBSD nvmecontolr(8) r329824.

nvmectl(8): fix subcommand usage.

nvmectl(8): Remove some wdc subcommands from man page.
- wdc drive-log
- wdc get-crash-dump
- wdc purge
- wdc purge-monitor

Typos.

use setprogname()/getprogname(), do not hardcode the prognam name in fixed
strings
 1.20.2.2  08-Apr-2020  martin Merge changes from current as of 20200406
 1.20.2.1  10-Jun-2019  christos Sync with HEAD
 1.26.4.2  03-Dec-2021  martin Pull up the following revisions, requested by msaitoh in ticket #1384:

sys/dev/pci/pcireg.h 1.152-1.154, 1.156-1.161
sys/dev/pci/pci_subr.c 1.222, 1.227-1.232 via patch
sys/dev/pci/nvme_pci.c 1.31
sys/dev/pci/pci.c 1.158, 1.163
sys/dev/pci/ppb.c 1.74

- When parsing Enhanced Allocation entries, use the correct calculation
for finding the next entry.
- Add 32.0GT/s to the list of pcie speeds (PCIe 5.x.).
- Add Some PCI config information:
- Lane Margining at the Receiver
- NVME admin interface
- UFSHCI
- InfiniBand
- Host fabric
- HDA 1.0 with vendor ext
- USB4 HCI
- MIPI I3C
- Cellular controller/modem (+ Ethernet)
- Change PCI_VENDOR_MASK and PCI_PRODUCT_MASK to unsigned values, to
prevent sign extension of product ID when shifted up into place in
PCI_ID_CODE(). Fixes PR kern/56176.
- Add LCAP & LCAP2 definitions.
- Use PCI-SIG official acronyms for some macros.
- Fix typo in some messages.
- Fix typo in comments.
- Whitespace fixes.
 1.26.4.1  21-Jun-2021  martin Pull up following revision(s) (requested by riastradh in ticket #1305):

sys/dev/ic/nvmevar.h: revision 1.22
sys/dev/ic/nvme.c: revision 1.56
sys/dev/ic/nvme.c: revision 1.57
sys/dev/pci/nvme_pci.c: revision 1.30

nvme(4): Add suspend/resume, derived from OpenBSD.

nvme(4): Move disestablishment of admin q interrupt to nvme_detach.

Nothing re-established this after suspend/resume, so attempting
suspend/resume/suspend would crash, and presumably we would miss
interrupts after resume. This keeps the establish/disestablish more
symmetric in attach/detach.
 1.29.8.1  31-May-2021  cjep sync with head
 1.29.6.1  17-Jun-2021  thorpej Sync w/ HEAD.

RSS XML Feed