Update ixg(4) and ixv(4) up to FreeBSD r292674:
- Add X552 SFP+ and X550T single port.
- Bug fixes.

Update ixg(4) and ixv(4) up to FreeBSD r282299:
- Add support for X55x.
- ADD EEE support (not tested).
- Add WOL support (not tested).
- Add suspend/resume support (not testd).
- Add LPLU support (not tested).
- Add DMA Coalescing (note verified).
- Not tested well for sysctls.
- Fix ixgbe_set_advertise() a bit. At least, FreeBSD r294578 is required
to work hw.ixg0.advertise_speed sysctl correctly.

Update ixg(4) and ixv(4) up to FreeBSD r280197:
- Add support for 82599_QSFP_SF_QP and X540T1.
- Add partial support for X55x. It's required to sync with FreeBSD r282299
to support X55x.
- Bugfixes.

branches: 1.2.2;
Sync ixg(4) up to FreeBSD r250108:
- Cleanup some unused counters and some unused code.
- Improve performance.
- Fix flow control - don't override user value on re-init
- Fix to make 1G optics work correctly
- Change to interrupt enabling - some bits were incorrect
for certain hardware.
- Certain stats fixes, remove a duplicate increment of
ierror, thanks to Scott Long for pointing these out.
- Fix the setting of RX which related to multicast.
- Some netmap related fixes.

branches: 1.1.12; 1.1.28; 1.1.30;
Add sources for ixgbe(4), a Intel 82599 10-gigabit ethernet driver
ported from FreeBSD.

Update ixg(4) and ixv(4) up to FreeBSD r292674:
- Add X552 SFP+ and X550T single port.
- Bug fixes.

branches: 1.2.2;
Sync ixg(4) up to FreeBSD r279805 (or r280181) which include some bugfixes.
TODO:
- Merge r280182 and newer. It's required to support X55x.
- MSI/MSI-X support.

branches: 1.1.12; 1.1.28; 1.1.30;
Add sources for ixgbe(4), a Intel 82599 10-gigabit ethernet driver
ported from FreeBSD.

ixgbe: Rename some definitions, modify comment. No functional change.

Apply changes from FreeBSD's ix-3.3.31 and ixv-1.5.32.
- struct adapter *adapter -> struct ixgbe_softc *sc
- master -> primary
- black -> block

branches: 1.9.4;
fix multiplei repetitive typos in comments, messages and documentation. mainly because copy paste code big amount of files are affected.

Add missing __KERNEL_RCSID().

Add NetBSD RCS IDs. No functional change.

branches: 1.6.6;
Fix checking return value of atomic_cas_uint().

This change fixes a bug that extra delay() is called only once even if
atomic_cas_uint() isn't failed or delay() isn't called when atomic_cas_uint()
failed.

The reason of this bug was that I simply converted FreeBSD' atomic_cmpset_int()
to atomic_cas_uint(). The return value's semantics is different.

No functional change:

- Remove unused code.
- Remove extra spaces.
- KNF.

branches: 1.4.2; 1.4.8;
Sync with the remaining part of FreeBSD r328265 except sfp_reinit stuff:
- Always schedule module intterrupt in ixgbe_config_link() when a device is
SFP+ based.
- Use not loop index but txr->me in ixv_initialize_{transmit,receive}_units().
It's required for VMDQ but NetBSD doesn't use it, so it's not a bug in
NetBSD.
- Simplify ixgbe_bp_wd_set(). No functional change.
- Whitespace.

Add some changes from ix-3.2.17.tar.gz and r328265. Not fully synchronized.
Some others (e.g. sfp cage interrupt and bypass adapter stuff) will be merged
later:
- Initialize firmware command buffer correctly in ixgbe_read_ee_hostif_X550()
and ixgbe_read_ee_hostif_buffer_X550(). These functions are used when reading
NVM.
- Fix a bug that ixgbe_mng_present() misunderstand management capability on
X550 and newer on some environment. X550 changed FWSM bit definition.
See X540 document and X550's document and compare them.
- Fix checksum calculation in ixgbe_set_fw_drv_ver_generic(). This function is
not used in NetBSD.
- Add some unused funtions.
- Whitespace fix.

branches: 1.2.2; 1.2.4; 1.2.6;
Fix a bug that bypass adapter's sysctls aren't set. Tested with non-genuine
X550-T2 bypass adapter:
- Call ixgbe_sysctl_instance() in ixgbe_bypass_init() to get sysctl's top node
correctly.
- ixgbe_init_device_features() refers adapter->hw.bus.func for bypass adapter.
Call set_lan_id() to set adapter->hw.bus.func before calling
ixgbe_init_device_features(). Without this, bypass sysctl's are added to
both the first and second port.
- Initalize node.sysctl_data before calling sysctl_lookup() to read correct
value.

- Sync with FreeBSD r320688 (and up to r322865):
- Add C3000(Denvertion(-NS)) support.
- Add bypass function support for bypass adapters. Sysctls are provided.
Not tested because I have no any bypass adapter.
- ixv(4): set RSS mapping.
- Change EEE sysctl.
- ixv(4): Add TSOv6.
- ixv(4): Mailbox API 1.2 and more are implemented and comment says it
negotiate with 1.2 but it really does 1.1...
- Remove thermal test sysctl.
- Fix unknown bugs.
- Print driver feature capabilities and enable bits when verbose boot.

ixgbe: Rename some definitions, modify comment. No functional change.

Apply changes from FreeBSD's ix-3.3.31 and ixv-1.5.32.
- struct adapter *adapter -> struct ixgbe_softc *sc
- master -> primary
- black -> block

branches: 1.5.14;
Add missing __KERNEL_RCSID().

Add NetBSD RCS IDs. No functional change.

branches: 1.3.6;
- Remove extra callout_stop() in ixgbe_detach(). Found by knakahara@.
- Rename ix{gbe,v}_free_workqueue() to ix{gbe,v}_free_deferred_handlers().
- Add KASSERT() to functions who are called from ixgbe_handle_admin().

branches: 1.2.8;
Sync with the remaining part of FreeBSD r328265 except sfp_reinit stuff:
- Always schedule module intterrupt in ixgbe_config_link() when a device is
SFP+ based.
- Use not loop index but txr->me in ixv_initialize_{transmit,receive}_units().
It's required for VMDQ but NetBSD doesn't use it, so it's not a bug in
NetBSD.
- Simplify ixgbe_bp_wd_set(). No functional change.
- Whitespace.

branches: 1.1.2; 1.1.4; 1.1.6;
- Sync with FreeBSD r320688 (and up to r322865):
- Add C3000(Denvertion(-NS)) support.
- Add bypass function support for bypass adapters. Sysctls are provided.
Not tested because I have no any bypass adapter.
- ixv(4): set RSS mapping.
- Change EEE sysctl.
- ixv(4): Add TSOv6.
- ixv(4): Mailbox API 1.2 and more are implemented and comment says it
negotiate with 1.2 but it really does 1.1...
- Remove thermal test sysctl.
- Fix unknown bugs.
- Print driver feature capabilities and enable bits when verbose boot.

ixgbe: Rename some definitions, modify comment. No functional change.

Apply changes from FreeBSD's ix-3.3.31 and ixv-1.5.32.
- struct adapter *adapter -> struct ixgbe_softc *sc
- master -> primary
- black -> block

branches: 1.17.4;
Add code to support API version 1.5. No functional change.

- This change adds almost all code to support API 1.5 except real negotiation
and upgrade mailbox functions.
- From ix-3.3.18, ix-3.3.22 and ixv-1.5.24.

Change the first argument of ixgbe_send_vf_msg(). No functional change.

- This is a part of FreeBSD ix-3.2.17.

Rename IXGBE_VT_MSGTYPE_{ACK,NACK} to IXGBE_VT_MSGTYPE_{SUCCESS,FAILURE}.

- Sync with FreeBSD ix-3.3.18.
- No functional change.

Move PF mailbox initialization from ixgbe_attach() to ixgbe_init_iov().

From FreeBSD ix-3.3.18. No functional change.

Change argument. No functional change. This file is not used in NetBSD.

Remove unused argument. Note that this file is not used in NetBSD.

Add missing __KERNEL_RCSID().

Add NetBSD RCS IDs. No functional change.

branches: 1.9.4;
s/boolean_t/bool/. boolean_t is obsolete (from Mach), and this
also fixes a new build issue in libkvm on sparc*.

branches: 1.8.2;
- Remove extra callout_stop() in ixgbe_detach(). Found by knakahara@.
- Rename ix{gbe,v}_free_workqueue() to ix{gbe,v}_free_deferred_handlers().
- Add KASSERT() to functions who are called from ixgbe_handle_admin().

Reduce ixgbe's busy loop using with workqueue and kpause.

- Use workqueue instead of softint to make some functions sleepable.
- Use new workqueue and enqueue it in ixgbe_local_timer() and
ixgbe_recovery_mode_timer() to make them sleepable.
- Make new ixgbe_delay() and use it. This functions sleeps if the time is
more than equals 1 tick. If it's not, do delay().

branches: 1.6.2;
Sync with FreeBSD ix-3.3.8 part 1. No functional change in this part:
- Move ixgbe_toggle_txdctl() to ixgbe_common.c and modify a bit.
No functional change because this function is currently used for SR-IOV
and it's not used in NetBSD.
- Some modification to match the latest netmap API.
- Modify ixgbe_hic_unlocked(). No functional change because neither
IXGBE_HOST_INTERFACE_APPLY_UPDATE_CMD(0x38) nor
IXGBE_HOST_INTERFACE_SHADOW_RAM_READ_CMD(0x31) are used.
- Add ixgbe_clear_mbx(). No functional change because this function is not
used yet.
- Add some not-yet-used register definitions.
- Whitespace fixes.

Apply FreeBSD ix-3.3.6.tar.gz's change to NetBSD. Tested on C3000 and X550-T1,
but not tested on Xeon D:
- Add firmware recovery mode for X550, X550A(Xeon D) and X550EM (C3000):
- FreeBSD always set IXGBE_FEATURE_RECOVERY_MODE without checking the
NVM image version. We compare it against 2.0 to not to make new callout and
not to call extra atomic operations.
- In some FreeBSD's sysctl functions, atomic_load_acq_int() is called
before a null pointer check. We call it after null pointer check.
- Before calling atomic_load_acq_uint(), check adapter->feat_en flags
to save atomic operation call.
- We don't check recovery_mode in ixgbe_set_sysctl_value() because this
function doesn't touch any hardware register.
- NetBSD don't have FreeBSD's atomic_load_acq_int()-like function, so do it
with membar_sync(). Thanks riastradh@ for the advice.
- FreeBSD's ix-3.3.6 changed ixgbe_enable_aim from TRUE to FALSE, but we will
keep it as TRUE because we have already fixed some bugs.
- Remove IXGBE_DEV_ID_82599_LS(0x154f) support again. I don't know why. This
was added in ix-3.2.18.tar.gz(NetBSD: ixgbe_82599.c rev. 1.20) and removed in
ix-3.3.6.tar.gz.
- On X550EMU, use ixgbe_identify_sfp_module_X550em() instead of
ixgbe_identify_module_generic(). ixgbe_identify_sfp_module_X550em() has
extra check (e.g. exclude 1G copper).
- if_sriov.c's change doesn't affect to NetBSD because we don't support
SR-IOV PF function.

Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

branches: 1.3.2;
Sync with the remaining part of FreeBSD r328265 except sfp_reinit stuff:
- Always schedule module intterrupt in ixgbe_config_link() when a device is
SFP+ based.
- Use not loop index but txr->me in ixv_initialize_{transmit,receive}_units().
It's required for VMDQ but NetBSD doesn't use it, so it's not a bug in
NetBSD.
- Simplify ixgbe_bp_wd_set(). No functional change.
- Whitespace.

Add some changes from ix-3.2.17.tar.gz and r328265. Not fully synchronized.
Some others (e.g. sfp cage interrupt and bypass adapter stuff) will be merged
later:
- Initialize firmware command buffer correctly in ixgbe_read_ee_hostif_X550()
and ixgbe_read_ee_hostif_buffer_X550(). These functions are used when reading
NVM.
- Fix a bug that ixgbe_mng_present() misunderstand management capability on
X550 and newer on some environment. X550 changed FWSM bit definition.
See X540 document and X550's document and compare them.
- Fix checksum calculation in ixgbe_set_fw_drv_ver_generic(). This function is
not used in NetBSD.
- Add some unused funtions.
- Whitespace fix.

branches: 1.1.2; 1.1.4; 1.1.6;
- Sync with FreeBSD r320688 (and up to r322865):
- Add C3000(Denvertion(-NS)) support.
- Add bypass function support for bypass adapters. Sysctls are provided.
Not tested because I have no any bypass adapter.
- ixv(4): set RSS mapping.
- Change EEE sysctl.
- ixv(4): Add TSOv6.
- ixv(4): Mailbox API 1.2 and more are implemented and comment says it
negotiate with 1.2 but it really does 1.1...
- Remove thermal test sysctl.
- Fix unknown bugs.
- Print driver feature capabilities and enable bits when verbose boot.

if_stats(9): Add ifp argument to if_stat..._ref.

This will enable us to pass the ifp through to a dtrace probe inside.

No functional change intended in this change, but this is an API
change visible to modules so it shouldn't be pulled up.

PR kern/58377

ixgbe: Use #ifdef RSC

This feature (hardware receive side coalescing) has been disabled all along,
so enclose the code with #ifdef RSC.

ixgbe: Use #ifdef LRO more to reduce the size of struct rx_ring.

ixgbe: Use kmem_zalloc() instead of malloc(,M_ZERO).

ixgbe: Simplify. No functional change.

The descriptor ring size and the alignment are tested in the attach
function, so it's not required to use roundup2(size, DBA_ALIGN).

ixgbe: Use #ifdef IXGBE_FDIR more

Don't include the Flow Director related members to reduce the size of
struct tx_ring. On amd64 and aarch64, the real size is not changed
because of the alignment.

ixgbe: micro-optimize ixgbe_txeof()

Update txr->packets outside the loop in ixgbe_txeof().

ixgbe: Update if_opackets outside the loop in ixgbe_txeof().

ixgbe: micro-optimize ixgbe_txeof()

Update txr->tx_avail and txr->txr_no_space outside the loop in ixgbe_txeof().

ixgbe: Modify for the readability. No functional change.

ixgbe(4): Modify comment. No functional change.

ixgbe_tx_ctx_setup() may or may not consume one TX descriptor.

ixgbe(4): Move assignment of TXD. NFCI.

ixgbe: Whitespace. No functional change.

ixgbe: Whitespace. No functional change.

ixgbe: Cleanup. No fucntional change.

Remove unused code, fix whitespace and modify comment to reduce against
FreeBSD(mainly from ix-3.3.35).

ixgbe: Rename some definitions, modify comment. No functional change.

Apply changes from FreeBSD's ix-3.3.31 and ixv-1.5.32.
- struct adapter *adapter -> struct ixgbe_softc *sc
- master -> primary
- black -> block

Remove obsolete comment.

branches: 1.100.4;
ixg(4) add an option for Tx to use deferred softint regardless of whether can get txq lock or not.

That imporve (7%) and stabilize throughput. But that can cause
latency degradation, so off by default.

ok'ed by msaitoh@n.o.

fix typos in comments.

bus_dmamem_unmap() before bus_dmamem_free(), otherwise we may give back meomry
which is still (and will stay) mapped.

Fixes one instance of "panic: HYPERVISOR_mmu_update failed" on Xen.
There may be others.

Use cached rx_copy_len in ixgbe_rxeof().

Modify comment for consistency. No functional change.

Use atomic_{load,store}_relaxed() for event counters.

Reduce bus_dmamap_sync() cost.

- Don't sync whole descriptor ring but limited counts(rx_process_limit)
descriptors.
- Dont' sync every loop.

Don't pre-allocate a cluster not to do m_freem() it on RXCOPY case.

Don't increment no_mbuf evcnt(9) when discarding multi-descriptor packet.

Sprinkle __predict_false() in the RX path.

Save the discard_multidesc state to not to forget the state by exiting rxeof().

Rename variable. No functional change.

Remove "j" or "jumbo" because we always use MCLBYTES sized buffer.

Use MCLGET() instead of homegrown cluster (jcl) allocation mechanism.

- Before this commit, resource shortage was easily occurred because the total
number of the clusters is small.

- Reviewed by knakahara and ryo.

Use m_adj(ETHER_ALIGN) more. Tested by me (amd64,aarch64) and rin (alpha).

- Align with ETHER_ALIGN everywhere where mbuf is allocated.
- Remove extra setting of M_PKTHDR. No functional change.
- Add comment.

KNF. No functional change.

Don't use fixed value. No functional change.

Don't call bus_dmamap_sync with rx_mbuf_sz(== MCLBYTES) to prevent panic.

If an mbuf is ETHER_ALIGNed, the packet size may be shorter than MCLBYTES.
For example, if the max frame size is 2048 by changing the interface's MTU,
an mbuf's max length is 2046. So, don't use rx_mbuf_sz for bus_dmamap_sync.

Refresh unrefreshed descriptors' buffers correctly.

- Update next_to_refresh at least before ixgbe_rx_unrefresed() to detect
the unrefreshed status correctly in ixgbe_rxeof().
- next_to_refresh points to the previous entry of the first unrefreshed
descriptor, so fix a loop variable to point to the correct one in
ixgbe_refresh_mbufs().
- Without the above two fixes, RX ring may have some unrefreshed entries
which have inconsistent state. On such state, "ifconfig down up" causes
panic in bus_dmamap_sync() on aarch64.
- Tested on amd64 and aarch64. OK'd by knakahara.

Add new sysctl "rx_copy_len".

ixgbe_rxeof() has an optimization "RX_COPY" to reduce costs of
bus_dmamap_load_mbuf() and bus_dmamap_unload() by copying a mbuf cluster's
memory to a newly allocated mbuf's MH_databuf[] and recycle the original map.
The optimization is used when a length of a packet is smaller than a specific
value. The value is calculated based on MHLEN. The size of MHLEN is
architecture specific. It's 256 or 512. Make the threshold controllable by
adding a new sysctl.

Set rxr->next_to_refresh correctly in ixgbe_setup_receive_ring().

ixgbe_setup_receive_ring() fully allocates rx buffers. When a
descriptor ring is full, rxr->next_to_refresh should point to
rxr_next_to_check -1. Before this change, rxr->next_to_refresh
is set to 0 and ixgbe_refresh_mbufs() wastefully loops in
ixgbe_refresh_mbufs() because it means the ring is empty.

Do no_jmbuf++ when ixgbe_getjcl() failed in ixgbe_setup_receive_ring(), too.

fix little-endian dependence

Use uint64_t instead of bus_addr_t for the TX descriptor's buffer address.
At least, this change is required for macppc (sizeof(bus_addr_t) == 4) to
make TX work.

Added BUS_DMA_COHERENT flag to bus_dmamem_map() to improve stability on aarch64.

In ixgbe, TX/RX descriptor rings are configured in 16-byte units.
If BUS_DMA_COHERENT is not specified, cpu cache (writeback/invalidate)
operations by bus_dmamap_sync() in aarch64 (arm/arm32/bus_dma.c) are done per
cache line size (usually 64 bytes). As a result, adjacent descriptors conflict
with the DMA operation, resulting in unstable operation.

To avoid this, descriptors area should be mapped as non-cache with BUS_DMA_COHERENT.


thanks to msaitoh@ for his help in debugging.

- Cleanup an rxbuf entry when bus_dmamap_load_mbuf() failed to prevent panic.
- Print error number when error occurred.

Keep m_len and m_pkthdr.len consistent to prevent panic on arm.

Arm's bus_dmamap_load_mbuf() keeps a pointer to the original mbuf
and bus_dmamap_sync() refers it. ixgbe_rxeof() modified mbuf's m_len
inconsistently with m_pkthdr and dm_mapsize. "ifconfig down up" made
panic by referring the inconsistent mbuf length.

Comment out flow director processing in fast path.

ixgbe_xmit() is per-queue fast path. It should reduce access to
per-device data (adapter->feat_en).

Call bus_dmamap_unload(9) via ixgbe_dmamap_unload(), before freeing
DMA buffer. Also, when the buffer is already freed, do not call
bus_dmamap_unload(9) (no resource leaks with this change).

Otherwise, MMU fault occurs for some bus_dma(9) implementations.

With this fix, X550-T1 and X540-T1 work fine on alpha (at least
DS10 with PCI-PCIe reverse bridge).

Discussed with msaitoh. Thanks!

branches: 1.71.2;
Add missing __KERNEL_RCSID().

branches: 1.70.2;
KNF a bit. No functional change.

branches: 1.69.2;
Refactor rxr->next_to_check updating.

Remove extra unlock/lock processing around if_percpuq_enqueue().

same as if_wm.c:r1.700

Modify some parameters to reduce packet dropping.

- Background: ixgbe doesn't use common MCLGET() interface and use the
driver specific cluster allocation mechanism (jcl). The cluster is
pre-allocated with a fixed number and the current number per queue
is num_rx_desc * 2 (2048*2=4096). It's too small. It also has a problem
that the max length of the pcq which is used in the TX path is big
(4096). Example:

100M <----- [ixg0 ixg1] <----- 1G
2048 TX descs <--- 4096 pcqs <---- 2048 RX descs

If a machine forwards a traffic from 1G interface to 100M interface,
It would require 2048+4096+2048=8192 descriptors, but the current number
is 2048*2=4096. It's too small. Even if the both interface's link speed
is the same and only small number of packet is queued in the pcq, 4096
jcl is small because 2048(RX)+TX(2048)=4096. If jcl is exhausted, not only
forwarding from ixg1 to ixg0 is dropped, but also another forwarding path
from ixg1 to another interface(e.g. wm0) is also dropped. Sockets also
queue packets, so if a lot of sockets are used and/or a socket buffer
size is changed to bigger one, it'll also become a problem. If the jcl
is exhausted, evcnt(9) counter "ixgX qY Rx no jumbo mbuf" is incremented.
Example:
vmstat -ev | grep ixg1 | grep "no jumbo"
ixg1 q0 Rx no jumbo mbuf 0 0 misc
ixg1 q1 Rx no jumbo mbuf 0 0 misc
ixg1 q2 Rx no jumbo mbuf 141326 0 misc
ixg1 q3 Rx no jumbo mbuf 0 0 misc


- To solve this problem:
- Add new config parameter IXGBE_JCLNUM_MULTI and set the default to 3
(2048 * 3). The minimum number is 2. The total number of jcl per queue
is available with hw.ixgN.num_jcl_per_queue sysctl.
- Reduce the max length of the pcq() which is used in the TX path from
4096 to 2048.

- Reviewed by knakahara@ and ozaki-r@.

- TODO: Use MCLGET().

Use no_jmbuf evcnt for the failure case of ixgbe_getjcl().

Fix jcl's starvation case in ixgbe_rxeof() again.

ix_txrx.c rev.1.64 preallocates jcl to prevent starvation but it's not
perfect. Don't use ixgbe_rx_discard() and just update the old descriptor
for reuse. It's also required for multiple descriptors case to refresh
subsequent descriptor(s). Reviewed by knakahara@.

Fix ixg(4) Rx interrupt stall when Rx buffers are exhausted.

Current ixgbe_rxeof() implementation calls ixgbe_refresh_mbufs()(and
ixgbe_getjcl()) after ixgbe_rx_input()(and if_percpuq_enqueue()).
And ixg(4) uses its own m_ext structure. That causes Rx interrupt
stall when Rx buffers are exhausted. e.g. TCP iperf3 with large
windows size. Furthermore, Rx descriptor problem has occurred
after stoppping the load.

To fix that problem, ixgbe_rxeof() must call ixgbe_getjcl() before
ixgbe_rx_input(), and must discard the new packet when ixgbe_getjcl()
fails.

By the way, ixg(4) should use MCLGET() instead of own m_ext structure.

ok'ed by msaitoh@n.o, thanks.

branches: 1.63.2;
No functional change:
- modify comment
- whitespace fix

branches: 1.62.4;
No functional change:

- Add debug printf()s.
- Remove unused macros.
- Remove extra newline.

Adopt <net/if_stats.h>.

Fix the freeing code for some error paths. Found and tested by Patrick Welche.

Free jumbo mem structure correctly. Found by yamaguchi@ using with LOCKDEBUG.

branches: 1.58.2;
No functional change:

- Remove unused code.
- Remove extra spaces.
- KNF.

in many device attach paths, allocate memory with M_WAITOK instead of M_NOWAIT
and remove code to handle failures that can no longer happen.

Fix missing kpreempt_disable() before softint_schedule() like if_vmx.c:r1.51.

printf -> device_printf

branches: 1.54.2;
Fix hung queue check when the queue number >= 31.

Sync with FreeBSD ix-3.3.8 part 1. No functional change in this part:
- Move ixgbe_toggle_txdctl() to ixgbe_common.c and modify a bit.
No functional change because this function is currently used for SR-IOV
and it's not used in NetBSD.
- Some modification to match the latest netmap API.
- Modify ixgbe_hic_unlocked(). No functional change because neither
IXGBE_HOST_INTERFACE_APPLY_UPDATE_CMD(0x38) nor
IXGBE_HOST_INTERFACE_SHADOW_RAM_READ_CMD(0x31) are used.
- Add ixgbe_clear_mbx(). No functional change because this function is not
used yet.
- Add some not-yet-used register definitions.
- Whitespace fixes.

Fix a bug that if_link_state_change(ifp, LINK_STATE_DOWN) isn't called unless
link goes up at least one time. Without this change, never linkup-ed media
keeps LINK_STATE_UNKNOWN instead of LINK_STATE_DOWN.

Apply the same fix as if_wm.c:r1.606 to ixg(4). Pointed out by msaitoh@n.o, thanks.

XXX pullup-8

Fix a bug that ixgbe_mq_start(an if_transmit function) returned wrong
vaue on error. pcq_put returns false on error, so returning it to caller
indicated no error.

XXX pullup-8

Make jcl allocation per queue to reduce mutex spin. Tested by me and
knakahara.

branches: 1.48.2;
Implement the BPF direction filter (BIOC[GS]DIRECTION). It provides backward
compatibility with BIOC[GS]SEESENT ioctl. The userland interface is the same
as FreeBSD.

This change also fixes a bug that the direction is misunderstand on some
environment by passing the direction to bpf_mtap*() instead of checking
m->m_pkthdr.rcvif.

Add "bool txr_no_space" for TX descriptor shortage. Use it like IFF_OACTIVE.

Don't check IFF_RUNNING in ixgbe_rxeof(). Doing break and leaving a deacriptor
with DD bit is worse than just processing the entry. And it's also racy to
check IFF_RUNNING in rxeof(). If you'd like to strictly obey IFF_RUNNING,
it would be better to do it in the upper layer.

Same as DragonFly (a part of 79251f5ebe4cf9dd2f3e6aed590e09d756d39922).

Revert new watchdog timer commits. The new watchdog timer made stability
worse than before. It seems unknown problems exists.

http://mail-index.netbsd.org/source-changes/2018/05/08/msg095020.html
http://mail-index.netbsd.org/source-changes/2018/05/16/msg095240.html

Whitespace fix. No functional change.

Fix a problem that the watchdog timer sometimes mistakenly fires. Restore
TX sending check in the end of ixgbe_txeof which was wrongly removed in
ix_txrx.c rev. 1.42.

- Fix broken watchdog timer. This change detects TX device timeout correctly.
NOTE: It's supporsed not to be called {ixgbe,ixv}_rearm_queues() in the
timer. Those are not required if any chip have no bug. In reality,
ixgbe_rearm_queues() is required on 82599 and newer chip AND other than
queue 0 to prevent device timeout. When it occured, packet was sent but the
descriptor's DD bit wasn't set even though IXGBE_TXD_CMD_EOP and
IXGBE_TXD_CMD_RS were set. After forcing interrupt by writing EICS register
in ixgbe_rearm_queues(), DD is set. Why? Is this an undocumented errata? It
might be possible not call rearm_queues on 82598 or queue 0, we call in any
cases in case the problem occurs. On ixv(4), I have not seen this problem yet
(though I tested only on X550_X(Xeon D 12xx)'s virtual function), but we
do rearm in case TX device timeout happen.
- ixv(4): Call callout_stop() earlier in ixv_stop() like ixgbe_stop().
- KNF.

Don't free and reallocate bus_dmamem when it's not required. Currently,
the watchdog timer is completely broken and never fire (it's from FreeBSD
(pre iflib)). If the problem is fixed and watchdog fired, ixgbe_init() always
calls ixgbe_jcl_reinit() and it causes panic. The reason is that
ixgbe_local_timer1(it includes watchdog function) is softint and
xgbe_jcl_reinit() calls bus_dmamem*() functions. bus_dmamem*() can't be called
from interrupt context.

One of the way to prevent panic is use worqueue for the timer, but it's
not a small change. (I'll do it in future).

Another way is not reallocate dmamem if it's not required. If both the MTU
(rx_mbuf_sz in reality) and the number of RX descriptors are not changed, it's
not required to call bus_dmamem_{unmap,free}(). Even if we use workque, this
change save time of ixgbe_init().

I have a code to fix broken watchdog timer but it sometime causes watchdog
timeout, so I don't commit it yet.

Remove unused structure entries. No functional change.

Sync with the remaining part of FreeBSD r328265 except sfp_reinit stuff:
- Always schedule module intterrupt in ixgbe_config_link() when a device is
SFP+ based.
- Use not loop index but txr->me in ixv_initialize_{transmit,receive}_units().
It's required for VMDQ but NetBSD doesn't use it, so it's not a bug in
NetBSD.
- Simplify ixgbe_bp_wd_set(). No functional change.
- Whitespace.

Avoid issues caused by sending old packets at next link-up time.

This modification consists by the following two parts.
- drain packets in if_snd queue or corresponding txr->txr_interq
when link_active == false in ifp->if_start(), ifp->if_transmit(),
and deferred Tx processing
- drain packets in if_snd queue and all of txr->txr_interq's
at link-down time

ok by msaitoh@n.o.

Don't write EIMC directly. It is required to manage with struct ix_queue status.

XXX pullup-8

Add some changes from ix-3.2.17.tar.gz and r328265. Not fully synchronized.
Some others (e.g. sfp cage interrupt and bypass adapter stuff) will be merged
later:
- Initialize firmware command buffer correctly in ixgbe_read_ee_hostif_X550()
and ixgbe_read_ee_hostif_buffer_X550(). These functions are used when reading
NVM.
- Fix a bug that ixgbe_mng_present() misunderstand management capability on
X550 and newer on some environment. X550 changed FWSM bit definition.
See X540 document and X550's document and compare them.
- Fix checksum calculation in ixgbe_set_fw_drv_ver_generic(). This function is
not used in NetBSD.
- Add some unused funtions.
- Whitespace fix.

Make some event counters MP safe. Now all of the event counters are
MP safe.

branches: 1.34.2;
ixg(4) supports workqueue poll mode, but not enabled by default yet. (that is, the default behavior is *not* changed)

At the time of high load near the wire rate, the turnaround time
of update/delete processing such as "ifconfig ixg0 inet XXX" or
"ifconfig ixg0 delete" is very long. The main reason is CPU
starvation caused by ixg(4)'s softint poll mode. ixg(4) uses
workqueue poll mode instead of softint poll mode, so that this
problem will be fix.

This change may cause performance issues, so it is not enabled
by default yet. Although there are that issues, if you want to use
workqueue poll mode, do "sysctl -w hw.ixgXX.txrx_workqueue=1" while
there is no traffic on the ixgXX.

ok by msaitoh@n.o.

Fix poll mode assumption breaking.

ixgbe_{enable,disable}_intr() forcibly enable/disable all interrupts
regardless of current state. That can break poll mode assumption,
that is, queue interrupts must not occur while polling Tx/Rx rings.

E.g. "ifconfig ixg0 delete && ifconfig ixg0 192.168.0.1" on heavy
load traffic can causes this issue.

This fix may have 1% or 2% performance impact at short packets.

XXX
ixgbe_rearm_queues() which is called only via watchdog can also break
this poll mode assumption because writing EICS casues interrupts
immediately when interrupt auto mask enabled.
We will fix it with other issues about watchdog later.

ok by msaitoh@n.o.

Fix a potential bug that TX/RX might stall when TX rate limit reached.
This change is almost the same as the RX rate limit bug fix in ixgbe.c
rev. 1.121. I've never got any stall, but this must be a bug.

Increment rxr->packets correctly in ixgbe_rxeof() to calculate ITR value
of AIM (Auto Interrupt Moderation) correctly. See also ixgbe.c rev. 1.124.

XXX pullup-8

- Fixes two problem:
1) RX may accesses freed area.
2) if_init() takes long time on many core machine.
Call ixgbe_jcl_reinit() not in ixgbe_setup_receive_ring() but in the
biginning of ixgbe_setup_receive_structures(). It was OK for pre multiqueue,
but it's not OK now because we support multiqueue.
- Fix comment of ixgbe_free_receive_ring().

branches: 1.29.2;
VLAN ID uses pkthdr instead of mtag now. Contributed by s-yamaguchi@IIJ.

I just commit by proxy. Reviewed by joerg@n.o and christos@n.o, thanks.
See http://mail-index.netbsd.org/tech-net/2017/09/26/msg006459.html

XXX need pullup to -8 branch

- Sync with FreeBSD r320688 (and up to r322865):
- Add C3000(Denvertion(-NS)) support.
- Add bypass function support for bypass adapters. Sysctls are provided.
Not tested because I have no any bypass adapter.
- ixv(4): set RSS mapping.
- Change EEE sysctl.
- ixv(4): Add TSOv6.
- ixv(4): Mailbox API 1.2 and more are implemented and comment says it
negotiate with 1.2 but it really does 1.1...
- Remove thermal test sysctl.
- Fix unknown bugs.
- Print driver feature capabilities and enable bits when verbose boot.

Sync with FreeBSD r316541:

> Fix a double free in ixgbe_rxeof()
>
> Submitted by: rstone
> MFC after: 1 week
> Differential Revision: https://reviews.freebsd.org/D10255

Fix the partial chain check in ixgbe_rx_discard(). This bug was addded in
rev. 1.33.

Increase total number of RX buffers on multiqueue.

branches: 1.24.2;
Fix a bug that number of input packet counted twice. This bug was added
since sys/net/if.c rev.1.368.
http://mail-index.netbsd.org/source-changes/2016/12/15/msg079882.html

Fix a bug that if_obytes and if_omcasts aren't counted. This bug was added
in ixgbe.c rev. 1.27 by me. Reported by Uwe Toenjes.

branches: 1.22.4;
Fix a problem that m_defrag() isn't called in if_transmit path. Now both
if_start and if_transmit do m_defrag() correctly. This change improves the
performance of TSO.

Fix a bug that TSO doesn't work correctly. This bug was added in
ix_txrx.c rev. 1.17.

Use percpuq.

- Both mbuf_defrag_failed and m_defrag_failed existed. Remove m_defrag_failed.
- rx_irq, dropped_pkts, morerx, moretx and txloops counters are uncounted, so
remove them.
- Count handleq in ixv_handle_que().
- Detach event counters correctly.
- Set some per-queue event names correctly.

TX multiqueue. If you want to disable it, enable IXGBE_LEGACY_TX
in ixgbe_netbsd.h

- Count tso_err again. It was accidentally removed in ixgbe.c rev. 1.28.
- ixgbe.h: Sort entries to reduce diff against FreeBSD.

Reduce diff against FreeBSD. No functional change.

Call ixgbe_free_transmit_buffers instead of ixgbe_free_transmit_structures()
in ixgbe_allocate_transmit_buffers()...

branches: 1.14.2; 1.14.4;
Fix INIT_DEBUGOUT() messages.

The rx_bytes event counter is cleared when if_init() is called, but not for
rx_packets. It's inconsistent. It makes rx_packets is bigger than rx_bytes.
IMHO, it's not required to clear them. At least the above comment says
"/* Setup our descriptor indices */". It's not descriptor indice...

Move bpf_mtap and if_ipackets++ on Rx of each driver to percpuq if_input

The benefits of the change are:
- We can reduce codes
- We can provide the same behavior between drivers
- Where/When if_ipackets is counted up
- Note that some drivers still update packet statistics in their own
way (periodical update)
- Moved bpf_mtap run in softint
- This makes it easy to MP-ify bpf

Proposed on tech-kern and tech-net

Fix ip[46]csum-tx doesn't work other than TCP and UDP.

branches: 1.10.2;
Update ixg(4) and ixv(4) up to FreeBSD r303890:
- Configure ixgbe phy & gbic power.
Setup phy and gbic power as per Linux 4.3.13 driver.
This fixes link not detected on X540-AT2 after booting to Linux which
turns the phy power off on detach. (FreeBSD r295093)
- Fixup DA cable detection routine. (FreeBSD r303032)
- Some minor changes

Update ixg(4) and ixv(4) up to FreeBSD r294578:
- Fixup SFP module insertion on the 82599 when insertion happens after
the system is booted and running. Add PHY detection logic to
ixgbe_handle_mod() and add locking to ixgbe_handle_msf() as well.
FreeBSD r293334.
- Fix ix advertise value after media change. When ifconfig sets media then the
values displayed by the advertise_speed value are invalidated.
Fix this by setting the bits correctly including setting advertise to 0 for
media = auto. FreeBSD r294578.
- Some others (e.g. LRO(not used by NetBSD)).

Update ixg(4) and ixv(4) up to FreeBSD r292674:
- Add X552 SFP+ and X550T single port.
- Bug fixes.

Update ixg(4) and ixv(4) up to FreeBSD r289238:
- Add support for sysctl knobs to live tune the per interrupt rx/tx packet
processing limits in ixgbe(4)
- Some others (netmap, etc.)

Update ixg(4) and ixv(4) up to FreeBSD r285590:
- Fix igxbe SRIOV VF (if_ixv) initialization bugs. The MAC address for
an if_ixv instance can now set at creation time, and the receive ring
tail pointer is correctly initialized (previously, things still worked
because the receive ring tail pointer was being fixed up as a side
effect of other activity).

Update ixg(4) and ixv(4) up to FreeBSD r283881:
- SRIOV support (not enabled because NetBSD doesn't support it).

Update ixg(4) and ixv(4) up to FreeBSD r282299:
- Add support for X55x.
- ADD EEE support (not tested).
- Add WOL support (not tested).
- Add suspend/resume support (not testd).
- Add LPLU support (not tested).
- Add DMA Coalescing (note verified).
- Not tested well for sysctls.
- Fix ixgbe_set_advertise() a bit. At least, FreeBSD r294578 is required
to work hw.ixg0.advertise_speed sysctl correctly.

Update ixg(4) and ixv(4) up to FreeBSD r280197:
- Add support for 82599_QSFP_SF_QP and X540T1.
- Add partial support for X55x. It's required to sync with FreeBSD r282299
to support X55x.
- Bugfixes.

Extra "buf->map = NULL;"s in ixgbe_txeof() were removed in FreeBSD r282299.
This was fixed in NetBSD's ixgbe.c rev. 1.28. Remove our comment.

FreeBSD r280182 made new file ix_txrx.c and moved ixgbe.c and ixv's common
code into it. Before sync with whole of them, just move ixgbe.c and ixv.c's
common code into ix_txrx.c from ixgbe.c. In this commit, only ixgbe.c is split
into the device dependent part and the common part. ixv.c isn't change to make
this commit no functional change. To use ixv.c with ix_txrx.c, it's required
to modify the common part's API and functions themselves.

This commit is done to make the next change easy to understand.

PR 59740: avoid use after free (and obvious crash if hardware
initialization fails)

ixgbe: Remove ifdef NET_MPSAFE. Make ixgbe MP-scalable by default.

Same as wm(4).

ixgbe: ixgbe is always MP-safe, so remove IXGBE_MPSAFE macro.

Before this change, IXGBE_MPSAFE is not defined if NET_MPSAFE is not set.
In this case, the PCI_INTR_MPSAFE flag is not set by pci_intr_setattr()
and interrupt_distribute() fails on arm. Always enable IXGBE_MPSAFE
funciton by removing the macro.

branches: 1.352.2;
if_stats(9): Add ifp argument to if_stat..._ref.

This will enable us to pass the ifp through to a dtrace probe inside.

No functional change intended in this change, but this is an API
change visible to modules so it shouldn't be pulled up.

PR kern/58377

Move RECOVERY_MODE feature test code for readability. No functional change.

ixgbe.c rev. 1.169 inserted RECOVERY_MODE feature test in between
MSI-X allocation and legacy allocation. To improve code readability,
move it to earlier location.

Modify comment. The number of queues is not limited to 8.

The number of queue is calculated from both the number of CPUs and
the number of MSI-X vectors.

ixgbe: Add QPRDC into iqdrops.

A receive packet might drop at two different locations.
One is the packet buffer that packets are received into the chip first.
If the packet buffer is overflowed, the MPC register is incremented.
It's currently added to iqdrops. It's no problem.
Another is descriptor ring(s). A packet from the packet buffer is DMA'ed
into main memory base on the descriptor ring. If the ring is full, the packet
is dropped and the QPRDC register is incremented. It should be added to
iqdrops but it was not done. Fix it.
Reported by ozaki-r@.

ixgbe: Clear the WTHRESH bit field before writing it.

Use ether_bpf_mtap only when the device supports vlan harware tagging

The function is bpf_mtap() for ethernet devices and *currently*
it is just handling VLAN tag stripped by the hardware.

ixgbe: Whitespace. No functional change.

ixgbe: Fix comment. No functional change.

ixgbe: Whitespace. No functional change.

ixgbe: Don't override the {ixgbe,ixv}_max_interrupt_rate global variable.

Fix a bug that changing hw.ix[gv]X.qY.interrupt_rate would change all
devices all queues default interrupt rate.

ixg(4): Don't print wrong error message about ixgbe_num_queues.

Don't override the ixgbe_num_queues global variable. It's the default
value of the number of queues and should not override it because it
will be referenced by later device attach. For example, the number of
MSI-X vector is 64 on X540 and 18 on 82599. When both cards are inserted
to a machine that the number of CPU is 24 and X540 is probed earlier,
ixgbe_num_queues is overridden to 24 and the following error message is
printed when attaching 82599:

ixg2: autoconfiguration error: ixgbe_num_queues (24) is too large,
using reduced amount (17).

Note that the number of queues is in sc->num_queuss and referenced
by hw.ixgN.num_queues sysctl.

ixgbe: Whitespace. No functional change.

ixg(4): Print DEVICE_CAPS register.

ixg(4): Whitespace. No functional change.

ixgbe: Cleanup. No fucntional change.

Remove unused code, fix whitespace and modify comment to reduce against
FreeBSD(mainly from ix-3.3.35).

ixgbe: Enable interrupt after setting IFF_RUNNING. Same as FreeBSD x-3.3.35.

ixgbe: Modify error message of wrong TX/RX descriptor size.

- Based from FreeBSD ix-3.3.35. I think ix-3.3.35's RING_INCREMENT(== 32)
is wrong. It should be 8(DBA_ALIGN / sizeof(union ixgbe_adv_[tr]x_desc)).
Linux also uses 8.

ixgbe: Simplify definitions. No functional change.

Both DEFAULT_{TX,RX}D and PERFORM_{TX,RX}D are 2048. Use DEFAULT_{TX,RX}D.
Same as FreeBSD.

ixg(4): Add 82599 LS once again.

- From:
FreeBSD: 9228ac3a69c4c7401a743e6465e118101a2beeb0
DPDK: 549ccd3dc01539e060597b503f2b65b272de3347
- This was removed 5 years ago. From the commit message:
> Remove IXGBE_DEV_ID_82599_LS(0x154f) support again. I don't know why. This
> was added in ix-3.2.18.tar.gz(NetBSD: ixgbe_82599.c rev. 1.20) and
> removed in ix-3.3.6.tar.gz.

ixgbe: Rename some definitions, modify comment. No functional change.

Apply changes from FreeBSD's ix-3.3.31 and ixv-1.5.32.
- struct adapter *adapter -> struct ixgbe_softc *sc
- master -> primary
- black -> block

ixg(4): Update FCTRL after writing multicast filter. Same as other OSes.

From FreeBSD 395cc55d896654b8f75071e71e856b22aed87da5.

ixg(4): Rename some descriptions of flow control related event conters.

ixg(4): Reorder some flow control related event counters for readability.

ixg(4): Count Queue Bytes {Transmit, Receive} counter.

ixg(4): Rename some descriptions of event counters.

- Rename some descriptions from register name to the meaning.
- For the same meaning's counters, add "(soft)" or "(reg)".
The former is for a software level counter and the latter is for a
statistics counter register based.

ixgbe: Reorder some event counters for readability.

Count the number of link down events in the MAC using with LINK_DN_CNT.

- Add new event counter "link_dn_cnt" to count the number of link down
events in the MAC.
- The LINK_DN_CNT register (at 0x0403c) is described only in the
Denverton's datasheet, so use it only on ixgbe_mac_X550EM_a.

Use thermal sensor code for IXGBE_DEV_ID_X550EM_A_10G_T, too.

PCI device ID 0x15c8 also use X557-AT PHY, so create the thermal sensor
sysctl for it, too.

branches: 1.324.2;
Make three "Unsupported SFP+ module..." messages the same.

Call txeof first, then rxeof for the consistency.

There are three functions where the txeof and rxeof are called. The legacy
interrupt function and MSI-X function call rxeof first, then rxeof.
For the workqueue function. rxeof is called first. Modify it to match with
other two.

aprintf_normal -> aprint_normal

Modify comment. ixgbe_enable_queue() can be used on both MSI-X and legacy intr.

KNF. Modify comment. No functional change.

KNF. No functional change.

Correctly enter the recovery mode. Not tested.

Correctly re-enable queue interrupt in ixgbe_legacy_irq().

- Don't enable queue 1-15 interrupt.
- Don't enable queue 0 interrupt when the queue work is being scheduled.
- OK'd by knakahara.

Simplify ixgbe_msix_que(). No functional change.

Fix a bug that the legacy interrupt doesn't work when MSI-X allocation failed.
Fixes PR kern/56857.

Add missing num_tx_desc sysctl.

Limit {tx,rx}_process_limit sysctl values from 1 to num_{tx,rx}_desc.

Print ECC, PHY and temp error log using with ratecheck().

- The ratecheck() is for just in case. All of the interrupts might occur
only once, but I don't know whether it's true or not. For the fan failure,
it seems it occurs only once.
- All of the interval is 60s.

Enclose flow director stuff in ixgbe_intr_admin_common() with IXGBE_FIR.

Move the definition of eicr_mask variable. No functional change.

Modify comment to clarify EIAC setting.

Add IFF_RUNNING check in ixgbe_legacy_irq() again.

- This change might fix small race between ifconfig down and an interrupt.
If the race really exists, txeof() is called and it's not so dangerous.
The rxeof is blocked in the beginning of ixgbe_handle_que().
- This change makes consistent with ixgbe_handle_que().
- Found and OK'd by knakahara.

Print Printed Board Assembly (PBA) number.

Increment legacy interrupt counter after checking INTx sharing.

Use atomic_{load,store}_relaxed() for event counters.

Use 64bit for lxon + lxoff.

Fix ierror counting again.

ixgbe.c rev. 1.298 added total values kept in evcnt(9) instead of
incremental values read from registers. Fix it.

sys: Use if_init wrapper function.

Exception: Not in kern_pmf.c, for the kind of silly reason that it
avoids having kern_pmf.c refer to symbols defined only in net; this
avoids a pain in the rump.

Add code to support API version 1.5. No functional change.

- This change adds almost all code to support API 1.5 except real negotiation
and upgrade mailbox functions.
- From ix-3.3.18, ix-3.3.22 and ixv-1.5.24.

Print NVM image version on 82598.

FreeBSD ix-3.3.29 added code to decode NVM version. On 82598, the NVM
offset is NVM_EEP_OFFSET_82598(== 0x2a). My own three different 82598
cards' value in NVM_EEP_OFFSET_82598 are 0xffff. Instead, the dev starter
version (0x29) has the value. Two of them have 0x1070 and another has 0x2090.
The 82598 specification update notes about 2.9.0.

Improve PHY's dmesg output for non-MIIVERBOSE case.

Add some missing error counters to ierror.

- FreeBSD: afb1aa4e6df245d38fd2ba683fa521d5dabe8392 or ix-ix-3.3.22
- Note that the checksum error is not added to the ierror.

Move PF mailbox initialization from ixgbe_attach() to ixgbe_init_iov().

From FreeBSD ix-3.3.18. No functional change.

No functional change.

- Sync with FreeBSD ix-3.3.18.
- Rename ixgbe_get_advertise() to ixgbe_get_default_advertise.
- Sort lines, modify comment and whitespace to reduce diff against FreeBSD.

fix typos in word "instead", mainly in log messages.

KNF

Remove vlan_ifdetach() from ixgbe_detach()

The removed function is called in ether_ifdetach().
And ether_ifdetach is changed to be called before a device
is stopped, because vlan_ifdetach called from ether_ifdetach
configures VLAN settings.

Fix a bug that an mbuf chain which has more than 63488bytes MAY fail on TX.

- Currently, The TX buffer dmamap's max number of segments is set to 32.
MCLBYTES(== 2048) * 32 = 65536 and it's enough for IP_MAXPACKET(65535).
If an mbuf chain has more than 32 mbufs, we call m_defrag() to make
it lower than equal to 32, but it might not work. The reason is that
our m_defrag() don't modify the first mbuf entry of the chain. e.g.:
if an mbuf chain contains 63600bytes data and the first mbuf has 100bytes
in the m_data, the new chain has 100+2048+2048+...+12 and the total number
of the chain is not 32 but 33. It result in 43 TCP packets will drop.
- One of the way to fix this problem is to change m_defrag() which add a
new mbuf cluster to the first mbuf. It's need discussion.
- Another solution is to change the max number of the TX DMA segment.
It should be at least 33 to avoid the m_defrag()'s current limitation.
The document (82599-X550 DS 7.2.1.1 "Transmit Storage in system Memory")
says that a packet can be fragmented into 40 - WTHRESH - 2 (for 82598,
the document say nothing and some people and code says it's unlimited).
Currently WTHRESH is set to 8. 40 - 8 - 2 = 30. !?!?!? {Net,Free,Open}BSD
and Linux use 32. Is that safe? Anyway, we change WTHRESH from 8 to 5
to fit it. The added comment in ixgbe.h is based on DragonFly's
though they don't use WTHRESH.
- Yet another solution is to use the Tx Head Pointer Write Back function
instead of WTHRESH based write back (see 82599-X550 DS 7.2.3.5.2
"Tx Head Pointer Write Back" or 82598 "Transmit Completions Head Write Back"
). DragonFly, illumos and DPDK use it.
- Yet yet another solution is to add tso_maxsize entry to struct ifnet
and use it in the TCP stack.

Use IXGBE_TXDCTL_WTHRESH_SHIFT macro. No functional change.

Remove "j" or "jumbo" because we always use MCLBYTES sized buffer.

Use MCLGET() instead of homegrown cluster (jcl) allocation mechanism.

- Before this commit, resource shortage was easily occurred because the total
number of the clusters is small.

- Reviewed by knakahara and ryo.

fix various typos in comments and log messages.

Add a new sysctl to read rxr->next_to_refresh.

Add new sysctl "rx_copy_len".

ixgbe_rxeof() has an optimization "RX_COPY" to reduce costs of
bus_dmamap_load_mbuf() and bus_dmamap_unload() by copying a mbuf cluster's
memory to a newly allocated mbuf's MH_databuf[] and recycle the original map.
The optimization is used when a length of a packet is smaller than a specific
value. The value is calculated based on MHLEN. The size of MHLEN is
architecture specific. It's 256 or 512. Make the threshold controllable by
adding a new sysctl.

Rework the xxxVERBOSE option to share the common module-hook-based
verbose mechanism with MIIVERBOSE. This reduces some duplicated code
and allows us to once again permit auto-unload of MIIVERBOSE.

Change details:
* Update dev/devlist2h.awk to accomodate miidevs, including generation
of MII_STR_oui_model definitions and use of oui and model rather than
vendor and product. This also changes the compressed data in the
xxxdevs_data.h files to uint32_t (since mii oui's are up to 6 hex
digits long)
* Update a couple of phy drivers to use new calls to get verbose data
* Regen all of the xxxdevs{,_data}.h files (separate commit, coming
very soon)
* Update mii/mii_verbose.[ch] and mii/mii_physubr.c to use the various
DEV_VERBOSE_xxx macros
* Update the pci, usb, and hdaudio code as needed, to #include the
xxxdevs.h files (in order to get the proper printf format strings)
* Since dev/dev_verbose.c now uses non-literal printf format strings,
(to deal with the vendor/product vs oui/model issue), we need to
make sure it gets compiled with -Wno-error=format-nonliteral, even
in userland's libpci and librumpdev!
* Bump kernel version for the change in module interfaces

Welcome to 9.99.86!

XXX It might be useful in the future to extend the MII_STR_oui_model
XXX definitions to PCI as well (and perhaps USB and HDAUDIO). This
XXX would allow for a single centralized location for the products'
XXX descriptions, rather than being dispersed among individual
XXX drivers' xxx_match tables.

if_attach and if_initialize cannot fail, don't test return value

These were originally made failable back in 2017 when if_initialize
allocated a softint in every interface for link state changes, so
that it could fail gracefully instead of panicking:

https://mail-index.NetBSD.org/source-changes/2017/10/23/msg089053.html

However, this spawned many seldom- or never-tested error branches,
which are risky to have around. And that softint in every interface
has since been replaced by a single global workqueue, because link
state changes require thread context but not low latency or high
throughput:

https://mail-index.NetBSD.org/source-changes/2020/02/06/msg113759.html

So there is no longer any reason for if_initialize to fail. (The
subroutine if_stats_init can't fail because percpu_alloc can't fail
either.)

There is a snag: the softint_establish in if_percpuq_create could
fail, potentially leading to bad consequences later on trying to use
the softint. This change doesn't introduce any new bugs because of
the snag -- if_percpuq_attach was already broken. However, the snag
can be better addressed without spawning error branches, either by
using a single softint or making softints less scarce.

(Separate commit will change the signatures of if_attach and
if_initialize to return void, scheduled to ride whatever is the next
convenient kernel bump.)

Patch and testing on amd64 and evbmips64-eb by maya@; commit message
soliloquy, and compile-testing on evbppc/i386/earmv7hf, by me.

- Cleanup an rxbuf entry when bus_dmamap_load_mbuf() failed to prevent panic.
- Print error number when error occurred.

branches: 1.282.2;
Print the error value of ixgbe_reset_hw() for debugging.

Add missing __KERNEL_RCSID().

branches: 1.280.2;
KNF a bit. No functional change.

branches: 1.279.2;
Modify some parameters to reduce packet dropping.

- Background: ixgbe doesn't use common MCLGET() interface and use the
driver specific cluster allocation mechanism (jcl). The cluster is
pre-allocated with a fixed number and the current number per queue
is num_rx_desc * 2 (2048*2=4096). It's too small. It also has a problem
that the max length of the pcq which is used in the TX path is big
(4096). Example:

100M <----- [ixg0 ixg1] <----- 1G
2048 TX descs <--- 4096 pcqs <---- 2048 RX descs

If a machine forwards a traffic from 1G interface to 100M interface,
It would require 2048+4096+2048=8192 descriptors, but the current number
is 2048*2=4096. It's too small. Even if the both interface's link speed
is the same and only small number of packet is queued in the pcq, 4096
jcl is small because 2048(RX)+TX(2048)=4096. If jcl is exhausted, not only
forwarding from ixg1 to ixg0 is dropped, but also another forwarding path
from ixg1 to another interface(e.g. wm0) is also dropped. Sockets also
queue packets, so if a lot of sockets are used and/or a socket buffer
size is changed to bigger one, it'll also become a problem. If the jcl
is exhausted, evcnt(9) counter "ixgX qY Rx no jumbo mbuf" is incremented.
Example:
vmstat -ev | grep ixg1 | grep "no jumbo"
ixg1 q0 Rx no jumbo mbuf 0 0 misc
ixg1 q1 Rx no jumbo mbuf 0 0 misc
ixg1 q2 Rx no jumbo mbuf 141326 0 misc
ixg1 q3 Rx no jumbo mbuf 0 0 misc


- To solve this problem:
- Add new config parameter IXGBE_JCLNUM_MULTI and set the default to 3
(2048 * 3). The minimum number is 2. The total number of jcl per queue
is available with hw.ixgN.num_jcl_per_queue sysctl.
- Reduce the max length of the pcq() which is used in the TX path from
4096 to 2048.

- Reviewed by knakahara@ and ozaki-r@.

- TODO: Use MCLGET().

Add "TX " to "Queue No Descriptor Available" to make it more understandable.

Reduce code duplication. No functional change.

Add new ixgbe_intr_admin_common() and use it in both ixgbe_msix_admin()
and ixgbe_legacy_irq().

Copy & paste some missing part (flow director, ECC and temp sensor) from
ixgbe_msix_admin() to ixgbe_legacy_irq(). Now it's ready to make a new
function to share the common part. It'll be done in near future.

Don't return in the middle of ixgbe_msix_admin() when an flow director reinit
failed. NetBSD currently doesn't support flow director, so this is not a real
bug.

Disable some interrupt in ixgbe_{legacy_irq,msix_admin}() to prevent log spam.

Disable/enable the OTHER interrupts correctly.

The OTHER interrupt was not blocked correctly when MSI-X is used.
ixgbe.c rev. 1.260 added new mutex to avoid the race but it didn't
disable the interrupt itself.

Calling ixgbe_enable_intr() enables all interrupts, so it's not good to
call it when some interrupt sources should not be enabled (e.g.:
calling ixgbe_enable_intr() in ixgbe_handle_admin() enables queue
interrupt).

IXGBE_REQUEST_TASK_NEED_ACKINTR doesn't work as expected because
ixgbe_handle_admin() can't know which task is enqueued from the
interrupt context and can't re-enable a specific EIMS bit.

Solve the above three problems by the following two changes:

- MSI-X: Disable the OTHER interrupts in the biginning of
ixgbe_msix_admin().

- Set mask bits correctly at the end of ixgbe_legacy_irq() and
ixgbe_msix_admin() using with eim_orig, eims_enable and eims_disable.

- Remove IXGBE_REQUEST_TASK_NEED_ACKINTR and add
IXGBE_REQUEST_TASK_{MOD,MSF}_WOI.

Check EICR's queue bit instead of IFF_RUNNING. This change fixes a bug that
it might incorrectly enable interrupt when IFF_RUNNING is not set. It also
changes that the TX/RX is not processed if a queue interrupt isn't occurred.

Don't use "more" flag for simplify in ixgbe_legacy_irq().
No functional change intended.

Simplify setting of EIAC register. No functional change intended.

Fix a bug that INTx is disabled if the INTx line is shared with other device.
ixgbe.c rev. 1.264 was not correct.

Restore EIMS before return. To read ECIR, clearing EIMC is required for
an errata, so
0) save the original EIMS value
1) clear EIMS
2) read EICR
3) restore with the saved value.

The EICR register are cleared in the beginning of the ixgbe_legacy_irq(),
so it's not required to clear each bit later in the function.

The EICR register's all OTHER interrupt bits are cleared in the beginning of
the ixgbe_msix_admin(), so it's not required to clear each bit later in the
function.

Reorder some code to reduce diff between ixgbe_legacy_irq() and
ixgbe_msix_admin. No functional change intended.

Add some debug printf()s and modify comments.

Remove strange ixgbe_enable_intr() in ixgbe_legacy_irq().

If the interface is UP and the INTx line is shared with other devices,
it result in enabling all interrupt sources even if some of them are
disabled for the workqueue. Delete ixgbe_enable_intr() in ixgbe_legacy_irq().

Correctly ACK the fan failure interrupt in ixgbe_legacy_irq().

Don't use EIMC_OTHER bit because it's read only other than 82598.

Documents say:

82598:
All of bit 31(OTHER bit) of EIxx are reserved. In reality, at least
EIMS_OTHER and EIMC_OTHER exist and the OTHER interrupt doesn't work
without EIMS_OTHER.

Other than 82598:
EIMS_OTHER is read only and EIMC_OTHER doesn't exist. If one of
bit 29..16 is set, EIMS_OTHER is set to 1 (Note that bit 30(TCP timer
isn't included)). Even if write bit 31 of EIMC to 1, it's ignored
(EIMS_OTHER doesn't set).

We introduced new spin mutex in ixgbe.c rev. 1.260, so it's OK to remove
EIMC_OTHER stuff. We already set EIMS_OTHER in if_init(), so keep it for
82598. No functional change other than 82598.

Another solution is to control bit 30..16 directly to mask/unmask interrupt
instead of the mutex.

TODO:
Some MSI-X interrupt(LSC, module insertion/removal etc.)'s mask/unmask
code between ixgbe_msix_admin() and ixgbe_handle_admin() may be wrong.
It'll be fixed later.

Apply ixgbe.c rev. 1.204 which was for ixgbe_msix_admin() to
ixgbe_legacy_irq(), too.

> An interrupt might not arrive when a module is inserted. When an link
> status change interrupt occurred and the driver still regard SFP as
> unplugged, issue the module softint before issuing LSC interrupt.

TODO: Reduce duplicated code.

Add new spin mutex to avoid race between ixgbe_msix_admin() and ixgbe_handle_admin().

At first, it seems "IXGBE_WRITE_REG(hw, IXGBE_EIMC, IXGBE_EIMC_OTHER)"
cannot stop interrupts, because 31th bit is reserved for 82598, 82599,
X540 and X550. So, the current following design
(1) ixgbe_msix_admin() disables interrupts
(2) ixgbe_msix_admin() calls workqueue_enqueue() for ixgbe_handle_admin()
(3) ixgbe_handle_admin() does interrupt processing
(4) after ixgbe_handle_admin() has done all interrupt processings,
ixgbe_handle_admin() enables interrupts
does not work correctly, that is, interrupts can be lost while
ixgbe_handle_admin() is running.

To fix that, add new spin mutex(adapter->admmin_mtx) which protects
atomically the following two members.
- adapter->admin_pending
- adapter->task_requests

The unnecessary "IXGBE_WRITE_REG(hw, IXGBE_EIMC, IXGBE_EIMC_OTHER)"
code will be removed later.

Reviewed and tested by hikaru@n.o and msaitoh@n.o, thanks.

Fix a bug that some advertise speeds can't be set with hw.ixgN.advertise_speed
if both 2.5G and 5G are set. Fix the error message, too.

branches: 1.258.2;
Fix race between ixgbe_msix_admin() and ixgbe_handle_admin(), pointed out by ozaki-r@n.o.

The race is caused by the following.
CPU#A processes workqueue, CPU#B processes admin interrupt.
(0) one of CPUs already calls ixgbe_schedule_admin_tasklet()
such as ixgbe_handle_timer()
(1) CPU#A: read adapter->task_requests
(2) CPU#B: set adapter->task_requests
(3) CPU#B: read(and try to set) adapter->admin_pending
but adapter->admin_pending is set, so does not
call workqueue_enqueue()
(4) CPU#A: clear adapter->admin_pending
that is, the tasks set by (2) is not processed as missfire workqueue by (3).

- Remove extra callout_stop() in ixgbe_detach(). Found by knakahara@.
- Rename ix{gbe,v}_free_workqueue() to ix{gbe,v}_free_deferred_handlers().
- Add KASSERT() to functions who are called from ixgbe_handle_admin().

__predict_false() should be __predict_true() for adapter->osdep.detaching
== false check.

Don't schedule admin work while shutdown like the recovery mode timer.

The admin workqueue also runs while !IFF_UP like the recovery mode timer.
Apply the same change of ixgbe.c 1.254 to ixgbe_schedule_admin_tasklet()
to prevent panic. Found by ozaki-r@.

Fix a panic on shutdown on a machine which use the recovery mode timer.

The recovery mode timer is first issued by the callout and it schedule
the workqueue. The workqueue then reschedule the callout. It's hard to
stop both of them without race only with callout_stop() and workqueue_wait.
To solve this problem. add new "detaching" flag and use it.

The situation is almost the same as schedule_wqs_ok for the local_timer's
callout and workqueue, but the difference is that the local_timer isn't
required to run if the interface is not up. If it's not important to prevent
running timer while !IFF_UP, the flag can be integrated into one.

Call ixgbe_ifstop() instead of ixgbe_stop_locked() in
ixgbe_setup_low_power_mode() to stop the timer workqueue.

Rename ix{gbe,v}_stop() with ix{gbe,v}_stop_locked(). No functional change.

If an SFP+ module is not inserted, don't try to access SFP+ EEPROM.
This change eliminate long timeout.

Reduce code duplication using with ixgbe_sfp_cage_full(hw).

Fix typo in comment.

Fix 82598 SFP+ problems.

On 82598, SFP+'s MOD_ABS isn't connected to the MAC's GPIO pin, so we can't
call ixgbe_sfp_cage_full(). Always issue TASK_MOD from ixgbe_handle_timer()
on 82598.

Fix ixgbe_identify_sfp_module_generic() for ixgbe_phy_nl. In the driver,
hw->phy.type sometimes be compared with ixgbe_phy_nl.
In ixgbe_identify_sfp_module_generic(), hw->phy.type may be overridden with
another value. For ixgbe_phy_nl, some code don't override phy.type but others
were not. Make it consistently keep ixgbe_phy_nl. This change fixes a problem
that ixgbe_is_sfp() change the return value true to false when any SFP+
devices are connected to the cage on 82598 and never recover from it.

Don't schedule MSF(multi speed fiber) task from ixgbe_handle_mod() on 82598.
This task is only for devices which support multi speed fiber and 82598
doesn't support it. Before ixgbe.c rev. 1.237, ixgbe_handle_mod() isn't
called on 82598 because 82598 has no SFP+ module insertion/removal interrupt.
ixgbe.c rev. 1.237 changed to call the function via timer on 82598.
This change fixes a bug that 82598 DA interface's link flaps.

Limit MA10-ST0's quirk only for on-chip devices.

Fix compile error.

Minor change.

- Print "X550EM X" instead of "X550EM" for Xeon D devices.
- Fix typo in comment. Same as FreeBSD.

ADD SFP+ MOD_ABS inversion quirk.

On X550 EM, GPIO(SDP) and SFP+'s MOD_ABS is directly connected. It has
no inverter. GIGABYTE MA10-ST0 has a inverter, so add new quirk for it.

Fix handling of IXGBE_REQUEST_TASK_NEED_ACKINTR again...

The admin workqueue can be used even if the interface is not up.
OK'd by thorpej@. Will fixes PR#55534 reported by Shinichi Doyashiki

Fix ixgbe_sfp_cage_full() on X550EM_A.

In ixgbe_handle_mod():

switch (hw->mac.type) {
case ixgbe_mac_82599EB:
cage_full = IXGBE_READ_REG(hw, IXGBE_ESDP) &
IXGBE_ESDP_SDP2;
break;
case ixgbe_mac_X550EM_x:
case ixgbe_mac_X550EM_a:
cage_full = IXGBE_READ_REG(hw, IXGBE_ESDP) &
IXGBE_ESDP_SDP0;
break;
default:
break;
}

so I had thought that IXGBE_ESDP_SDP0 bit is 1 on cage is full.
In reality, at least, X550EM_A's SFP+ cage is 0 on cage is full.
So invert the logic of ixgbe_sfp_cage_full() on X550EM_A

Fix race in ixgbe_detach() to prevent panic on shutdown.

Fix handling of IXGBE_REQUEST_TASK_NEED_ACKINTR in ixgbe_handle_admin().

Simplify SFP+ check. No functional change.

Re-enabling interrupt is required only when a work is scheduled form the
interrput context.

Fix a bug that the driver sometimes missed module insertion.

The ixgbe_sfp_probe() function was only for 82598 and other chips had no
way to poll SFP+ cage. The ixgbe_handle_mod() already has function to treat
module insertion/removal for all chips, so enqueue the work if the cage
status changed. All of ixgbe chips' SFP+ module interrupt is only on the
inserstion. This change also detect the removal by the timer.

Add missing workqueue_wait() for the recovery_mode_timer workqueue.

Set recovery_mode_timer workqueue's name correctly.

Use atomic_cas_uint() and atomic_store_relaxed(). Advised by thorpej@.
Tested by me. OK'd by knakahara.

Reduce ixgbe's busy loop using with workqueue and kpause.

- Use workqueue instead of softint to make some functions sleepable.
- Use new workqueue and enqueue it in ixgbe_local_timer() and
ixgbe_recovery_mode_timer() to make them sleepable.
- Make new ixgbe_delay() and use it. This functions sleeps if the time is
more than equals 1 tick. If it's not, do delay().

Modify a liitle to reduce diff between ixgbe.c and ixv.c. No functional change.

Add missing callout_stop() for recovery_mode_timer.

NOTE: One of the difference between the local_timer and recovery_mode_timer
is that local_timer runs only when IFF_UP and recovery_mode_timer always runs
(it's enabled on attach).

- Remove extra kpreempt_disable() -> kpreempt_enable() because
ixgbe_handle_msf() was changed from softint to workqueue.
- Set schedule_wqs_ok before enabling interrupt to prevent the race.
- Fix comment.

Fix typo. No functional change.

No functional change:
- modify comment
- whitespace fix

branches: 1.227.2;
Define and implement a locking protocol for the ifmedia / mii layers:
- MP-safe drivers provide a mutex to ifmedia that is used to serialize
access to media-related structures / hardware regsiters. Converted
drivers use the new ifmedia_init_with_lock() function for this. The
new name is provided to ease the transition.
- Un-converted drivers continue to call ifmedia_init(), which will supply
a compatibility lock to be used instead. Several media-related entry
points must be aware of this compatibility lock, and are able to acquire
it recursively a limited number of times, if needed. This is a SPIN
mutex with priority IPL_NET.
- This same lock is used to serialize access to PHY registers and other
MII-related data structures.

The PHY drivers are modified to acquire and release the lock, as needed,
and assert the lock is held as a diagnostic aid.

The "usbnet" framework has had an overhaul of its internal locking
protocols to fit in with the media / mii changes, and the drivers adapted.

USB wifi drivers have been changed to provide their own adaptive mutex
to the ifmedia later via a new ieee80211_media_init_with_lock() function.
This is required because the USB drivers need an adaptive mutex.

Besised "usbnet", a few other drivers are converted: vmx, wm, ixgbe / ixv.

mcx also now calls ifmedia_init_with_lock() because it needs to also use
an adaptive mutex. The mcx driver still needs to be fully converted to
NET_MPSAFE.

Ensure we don't call workqueue_enqueue() if the pluggable optics
handler is already pending.

No functional change:

- Add debug printf()s.
- Remove unused macros.
- Remove extra newline.

Update comment in ixgbe_update_stats_counters() to clarify why we update
if_iqdrops and if_ierrors only. No functional change.
OK'd by knakahara and thorpej.

- Fix locking problem with optics module interrupts: ifmedia_add()
may block on memory allocation, and so it cannot be safely done from
a softint nor can it be done while holding a spin lock. Fix this by
using a workqueue rather than a softint, and hold the IFNET_LOCK
across the entire handler, and the CORE_LOCK only across the code that
needs to serialize access to the hardware state.
- Use ifmedia_fini().

Tested in a variety of devices by msaitoh@. (Thanks!)

Adopt <net/if_stats.h>.

XXX This driver needs some work in this regard (practually no
stats are reported).

Fix the freeing code for some error paths. Found and tested by Patrick Welche.

branches: 1.220.2;
Fix some typos in comments.

From vezhlys on freenode IRC.

Add recovery code for unsupported SFP+.

Before this commit:
If an unsuppored SFP module is inserted before booting, the driver attach
failed and there was no way to recover form it without rebooting or
detaching/reattaching drvier (drvctl -d && drvctl -r pciN).
After this commit:
We can automatically recover any time by replacing it with a supported
module.

Add missing core lock in ixgbe_handle_mod().

Use bus_space_barrier() instead of x86 specific *fence instruction.
Written by riastradh@.

Print MAC address.

XXX Should we move such type of printf() to ether_ifattach?

in many device attach paths, allocate memory with M_WAITOK instead of M_NOWAIT
and remove code to handle failures that can no longer happen.

Apply FreeBSD r353599:

> ixgbe: Disable EEE for backplane X550EM_X
>
> From Zach:
> Intel documentation indicates that backplane X550EM_X KR devices do not
> support Energy Efficient Ethernet. Prior to this patch, X552 devices
> (device ID 0x15AB) will crash the system when transitioning EEE state
> via sysctl.
>
> Signed-off-by: Zach Vargas <zvargas@xes-inc.com>
>
> PR: 240320
> Submitted by: Zach Vargas <zvargas@xes-inc.com>
> Reviewed by: erj@
> MFC after: 3 days
> Differential Revision: https://reviews.freebsd.org/D21673

s/ixgbe_set_multi/ixgbe_set_rxfilter/. No functional change.

Remove ixgbe_set_promisc() and use ixgbe_set_multi() to reduce code
duplication.

Fix a bug that the multicast filter isn't correctly initialized when
the total number of the Ethernet multicast addresses is just 128.
Changing "<" with "<=" fixes this problem, but it's better to check
ETHER_F_ALLMULTI than it because it's straightforward.

if_flags is neither int nor short. It's unsigned short.

Fix a bug that MBSDC (Bad SFD Count) isn't counted on X550EM_X and X550EM_A.
The register is for X550 and newer.

Set IFM_1000_BX10 correctly.

printf -> device_printf

Fix panic when ncpu == 1 && sysctl -w hw.ixg0.txrx_workqueue=1. Reported by nonaka@n.o.

ok by msaitoh@n.o and nonaka@n.o

pullup-8, pullup-9

fix typo in error handling of ixgbe_allocate_legacy().

ok by msaitoh@n.o

pullup-8, pullup-9

An interrupt might not arrive when a module is inserted. When an link status
change interrupt occurred and the driver still regard SFP as unplugged, link
becomes up and the real media type is unknown. e.g:

% ifconfig -m ixg0
(snip)
media: Ethernet autoselect (autoselect rxpause,txpause)
status: active
supported Ethernet media:
media none
media autoselect
(snip)

To resolve this problem, when an link status change interrupt occurred and the
driver still regard SFP as unplugged, issue the module softint before issuing
LSC interrupt.

X550EM supports QSFP, so check ixgbe_media_type_fiber_qsfp too.

Simplify ix{gbe,v}_[un]register_vlan() API suggestesd by knakahara.

The API was the same as FreeBSD's pre-iflib's. They use iflib now and it's
not required for us to keep the old API.

Use ETHER_LOCK()/ETHER_UNLOCK() suggested by knakahara.

Fix a bug that VLAN HW "tagging" enable/disable may not refrect correctly.

- Always call ec_vlan_cb() if it exists.
- Some (or all?) ethernet drivers don't enable HW tagging if no any vlan is
attached. ixgbe is one of them. Check the the transition and update
VLAN HW tagging function.

XXX pullup-9

branches: 1.199.2;
ixgbe_setup_low_power_mode() calls ixgbe_stop() in it. ixgbe_detach() first
called ixgbe_stop() and then called ixgbe_setup_low_power_mode() which resulted
in calling ixgbe_stop() twice. Call only ixgbe_setup_low_power_mode() first to
remove extra ixgbe_stop() call.

Set IVAR (Interrupt Vector Allocation Register) correctly for 82598 in
ixgbe_set_ivar(). Found by KUBSan.

Simplify code. No functional change intended.

Fix VLAN hardware filter initialization. Use unsigned. Found by KUBSan.

Fix VLAN filter setting. Found by KUBSan.

Fix some problems found by KUBSan:
- Set TQSM and RQSM (TX/RX queue statistics mapping) correctly.
- Set IVAR (Interrupt Vector Allocation Register) correctly in
ixgbe_set_ivar().
- Define IXGBE_KRM_LINK_CTRL_1_TETH_AN_RESTART correctly. This macro is used
in ixgbe_restart_an_internal_phy_x550em().
- Define IXGBE_KRM_TX_COEFF_CTRL_1_OVRRD_EN correctly. This macro is used
in ixgbe_setup_ixfi_x550em_x().
- Define IXGBE_SB_IOSF_CTRL_BUSY correctly. This macro is used in
ixgbe_iosf_wait() (X550EM only).

Implement VLAN hardware filter function(ETHERCAP_VLAN_HWFILTER).
First proposed by jmcneill in 2017 and modified by me.

How to use:

- Set callback function:

ether_set_vlan_cb(struct ethercom *, ether_vlancb_t)

- Callback. This function is called when a vlan is attached/detached to the
parent interface:

int (*ether_vlancb_t)(struct ethercom *ec, uint16_t vlanid, bool set);

- ifconfig(8)

ifconfig ixg0 [-]vlan-hwfilter

Note that ETHERCAP_VLAN_HWFILTER is set by default on ixg(4) because
the PF driver usually enable "all block" filter by default.

On ENETRESET case, not continue and quit the ifflags_cb() function because
if_init() will do the same thing.

Sync with FreeBSD ix-3.3.10 part 2:
- Calculate vector's bit location correctly when the vector >= 31
in ixgbe_allocate_msix().

Sync with FreeBSD ix-3.3.10 part 1. No functional change in this part:
- "(u64)1" -> "1ULL"
- Add some not-yet-used register definitions.

Sync with FreeBSD ix-3.3.8 part 1. No functional change in this part:
- Move ixgbe_toggle_txdctl() to ixgbe_common.c and modify a bit.
No functional change because this function is currently used for SR-IOV
and it's not used in NetBSD.
- Some modification to match the latest netmap API.
- Modify ixgbe_hic_unlocked(). No functional change because neither
IXGBE_HOST_INTERFACE_APPLY_UPDATE_CMD(0x38) nor
IXGBE_HOST_INTERFACE_SHADOW_RAM_READ_CMD(0x31) are used.
- Add ixgbe_clear_mbx(). No functional change because this function is not
used yet.
- Add some not-yet-used register definitions.
- Whitespace fixes.

Match the follwing devices:

82598_BX (0x1508)
82599_KR (0x1517)
82599_SFP_EM (0x1507)
X550EM_X_XFI (0x15b0)
X550EM_A_QSFP (0x15ca)
X550EM_A_QSFP_N(0x15cc)

Even if we don't use MII(4), use the common path of SIOC[GS]IFMEDIA in
sys/net/if_ethersubr.c if we can.
- Add ec_ifmedia into struct ethercom.
- ec_mii in struct ethercom is kept and used as it is. It might be used in
future. Note that some Ethernet drivers which _DOESN'T_ use mii(4) use
ec_mii for keeping the if_media. Those should be changed in future.

Whitespace fix (mainly tabify).

-No functional change:
- KNF
- u_int*_t -> uint*_t.

Use new media types.

Store IFF_ALLMULTI in ec_flags instead of if_flags to avoid data races

IFF_ALLMULTI is set/unset to if_flags via if_mcast_op. To avoid data races on
if_flags, IFNET_LOCK was added for if_mcast_op. Unfortunately it produces
a deadlock so we want to remove added IFNET_LOCK by avoiding the data races by
another approach.

This fix introduces ec_flags to struct ethercom and stores IFF_ALLMULTI to it.
ec_flags is protected by ETHER_LOCK and thus IFNET_LOCK is no longer necessary
for if_mcast_op. Note that the fix is applied only to MP-safe drivers that
the data races matter.

In the kernel, IFF_ALLMULTI is set by a driver and used by the driver itself.
So changing the storing place doesn't break anything. One exception is
ioctl(SIOCGIFFLAGS); we have to include IFF_ALLMULTI in a result if needed to
export the flag as well as before.

A upcoming commit will remove IFNET_LOCK.

PR kern/54189

Remove unnecessary checks of IFF_ALLMULTI

IFF_ALLMULTI is changed by only driver itself, so we don't need to check its
change on ec_ifflags_cb.

This is part of PR kern/54189. NFCI.

Fix a bug that manual setting of 10GBASE-SR or 10GBASE-CX4 didn't work
if IFM_ETH_XTYPE was set. We don't use IFM_ETH_XTYPE macro, so this was not a
real bug.

Fix typo. This code is not used yet.

s/pakcet/packet/ in comment.

- Simplily. Suggested by knakahara.
- Modify comment. Per queue VLAN enable flags is on 82599 and later.
- Fix typo in comment.

- Fix a bug that the VLAN HW tagging function is not correctly disabled
when all vlan is detached.
- Fix a bug that VLAN HW tagging function is not correctly controlled on 82598.
- Control VLAN HW filter function correctly.
- Don't clear IXGBE_VLNCTRL_CFIEN bit When ETHERCAP_VLAN_HWFILTER is set.
I think it's not required (and Linux doesn't do it). This change has no
effect to NetBSD because ETHERCAP_VLAN_HWFILTER is not supported yet.

It's not required to calculate unused queues' statistics.

NetBSD currently uses traffic class 0 only. Other traffic classes aren't used
yet. When IXGBE_TC_COUNTER_NUM is set to lower than IXGBE_DCB_MAX_TRAFFIC_CLASS
(e.g. 1), other traffic classes' counters are not used. It means we don't
generate evcnt for them and don't add the values in
ixgbe_update_stats_counters().

Fix a bug that if_link_state_change(ifp, LINK_STATE_DOWN) isn't called unless
link goes up at least one time. Without this change, never linkup-ed media
keeps LINK_STATE_UNKNOWN instead of LINK_STATE_DOWN.

- Move the location of calling setup_eee() in ixgbe_attach(). The previous
location was too early. The new location is before calling
ixgbe_set_phy_power().
- Restore call of setup_eee() in ixgbe_init_locked(). It was removed in
rev. 1.99 (FreeBSD r320688).
- When calling setup_eee(), pass enable/disable correctly.

Call ixgbe_init_swfw_semaphore(hw) before calling ixgbe_check_wol_support()
because ixgbe_check_wol_support()->ixgbe_check_wol_support()->
ixgbe_get_device_caps() accesses semaphore.

Nowadays some UEFI BIOSes don't enable some PCI devices' address decoding.
To resolve this problem, pci_map.c rev. 1.34-1.36 changed the
pci_mapreg_(sub)map()'s to set the decode bit if it's not set. It's good for
almost all drivers, but some other drivers don't use pci_mapreg_map().
In drivers which don't use pci_mapreg_map(), some of them expilicitly enable
decoding but others don't. Add code to enable decoding to them.

See also the following discussion:
http://mail-index.netbsd.org/tech-kern/2017/03/22/msg021678.html

Don't call callout_halt() if it's not initialzied.

Apply FreeBSD ix-3.3.6.tar.gz's change to NetBSD. Tested on C3000 and X550-T1,
but not tested on Xeon D:
- Add firmware recovery mode for X550, X550A(Xeon D) and X550EM (C3000):
- FreeBSD always set IXGBE_FEATURE_RECOVERY_MODE without checking the
NVM image version. We compare it against 2.0 to not to make new callout and
not to call extra atomic operations.
- In some FreeBSD's sysctl functions, atomic_load_acq_int() is called
before a null pointer check. We call it after null pointer check.
- Before calling atomic_load_acq_uint(), check adapter->feat_en flags
to save atomic operation call.
- We don't check recovery_mode in ixgbe_set_sysctl_value() because this
function doesn't touch any hardware register.
- NetBSD don't have FreeBSD's atomic_load_acq_int()-like function, so do it
with membar_sync(). Thanks riastradh@ for the advice.
- FreeBSD's ix-3.3.6 changed ixgbe_enable_aim from TRUE to FALSE, but we will
keep it as TRUE because we have already fixed some bugs.
- Remove IXGBE_DEV_ID_82599_LS(0x154f) support again. I don't know why. This
was added in ix-3.2.18.tar.gz(NetBSD: ixgbe_82599.c rev. 1.20) and removed in
ix-3.3.6.tar.gz.
- On X550EMU, use ixgbe_identify_sfp_module_X550em() instead of
ixgbe_identify_module_generic(). ixgbe_identify_sfp_module_X550em() has
extra check (e.g. exclude 1G copper).
- if_sriov.c's change doesn't affect to NetBSD because we don't support
SR-IOV PF function.

Some counters are not per queue but per traffic class. Make new evcnt group
"ixgM tcN" (N = 0..7) and move those counters into it. We are using only
traffic class 0, so we will see only tc0's counter are updated.

Don't set LRO capability flags because we don't support it yet.
This doesn't change any behavior other than if_capabilities' visibility
(e.g. ifconfig).

- Fix a bug that Denverton accessed wrong register in
ixgbe_setup_low_power_mode(). The location of some registers in Denverton
were changed. The GRC register is one of them, so use IXGBE_GRC_BY_MAC(hw)
instead of IXGBE_GRC.
- ixgbe_check_wol_support() tests the capability of WOL, but the sysctl
knobs for WOL were added only for IXGBE_DEV_ID_X550EM_X_10G_T. Add the
knobs when adapter->wol_support is not 0.

XXX pullup-8

Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

Fix a bug that media change may fail. I noticed that ifconfig ixgN media XXX
took a 10 or more seconds on a Denverton machie. I occurred by trying take a
lock and timed out. The reason was that ixgbe_media_change() didn't take
CORE_LOCK. Do it.

This problem was from FreeBSD's pre-iflib ixgbe (I don't know whether this
problem really occurs on FreeBSD or not). Post-iflib ixgbe has no problem
because iflib_media_change() takes a lock.

XXX pullup-8

Add force_10_100_autonego sysctl. The default vaule is 0(false).
This sysctl is only for 550EM_a with PHY firmware for a while.

Remove nmbclusters check. We don't use the mbuf cluster. The old code also had
a bug that ixgbe_total_ports adds two every port and never decrement in
the detach path. Found by hikaru@.

The code was removed in FreeBSD when it switched to use iflib and OpenBSD
removed the code 8 years ago.

branches: 1.161.2;
Comment out unused functions in order to fix Clang build

ixv_rearm_queues() and ixgbe_rearm_queues() are unused.

Sponsored by <The NetBSD Foundation>

Don't call ixgbe_rearm_queues() in ixgbe_local_timer1().

ixgbe_enable_queue() and ixgbe_disable_queue() try to enable/disable queue
interrupt safely. It has the internal counter. When a queue's MSI-X is
received, ixgbe_msix_que() is called (IPL_NET). This function disable the
queue's interrupt by ixgbe_disable_queue() and issues an softint.
ixgbe_handle() queue is called by the softint (IPL_SOFTNET), process TX, RX
and call ixgbe_enable_queue() at the end.

ixgbe_local_timer1() is a callout and run always on CPU 0 (IPL_SOFTCLOCK).
When ixgbe_rearm_queues() called, an MSI-X interrupt is issued for a specific
queue. It may not CPU 0. If this interrupt's ixgbe_msix_que() is called and
sofint_schedule() is called before the last sofint's softint_execute() is not
called, the softint_schedule() fails because of SOFTINT_PENDING. It result
in breaking ixgbe_{enable,disable}_queue()'s internal counter.

ixgbe_local_timer1() is written not to call ixgbe_rearm_queues() if
the interrupt is disabled, but it's called because of unknown bug or a race.
One solution to avoid this problem is to not to use the internal counter,
but it's little difficult. Another solution is stop using
ixgbe_rearm_queues() at all. Essentially, ixgbe_rearm_queues() is not
required (it was added in ixgbe.c rev. 1.43 (2016/12/01)).
ixgbe_rearm_queues() helps for lost interrupt problem but I've never seen it
other than ixgbe_rearm_queues() problem.

XXX pullup-8.

Constify several variables in ixgbe/ so that they land in .rodata (1038
bytes).

Add hw.ixgN.debug sysctl. "sysctl -w hw.ixgN.debug=1" dumps some registers
to console.

Clear que->disabled_count in {ixgbe,ixv}_init_locked(). Without this,
interrupt mask state and EIMS may mismatch and if_init doesn't recover
from TX/RX stall problem.

This change itself doesn't fix PR#53294.

Ensure to call if_register after interface initializations finish

Add "bool txr_no_space" for TX descriptor shortage. Use it like IFF_OACTIVE.

Add rxd_nxck (Receive Descriptor next to check) read only sysctl.

Revert new watchdog timer commits. The new watchdog timer made stability
worse than before. It seems unknown problems exists.

http://mail-index.netbsd.org/source-changes/2018/05/08/msg095020.html
http://mail-index.netbsd.org/source-changes/2018/05/16/msg095240.html

The TQSMR register is not for receiving but for transmitting, so move the
initialization from ixgbe_initialize_receive_units() to
ixgbe_initialize_transmit_units(). No functional change.

Count some register correctly:
- QPRDC register is only for 82599 and newer.
- Count IXGBE_QPRDC, PX{ON,OFF}{T,R}XC[NT].

- Fix broken watchdog timer. This change detects TX device timeout correctly.
NOTE: It's supporsed not to be called {ixgbe,ixv}_rearm_queues() in the
timer. Those are not required if any chip have no bug. In reality,
ixgbe_rearm_queues() is required on 82599 and newer chip AND other than
queue 0 to prevent device timeout. When it occured, packet was sent but the
descriptor's DD bit wasn't set even though IXGBE_TXD_CMD_EOP and
IXGBE_TXD_CMD_RS were set. After forcing interrupt by writing EICS register
in ixgbe_rearm_queues(), DD is set. Why? Is this an undocumented errata? It
might be possible not call rearm_queues on 82598 or queue 0, we call in any
cases in case the problem occurs. On ixv(4), I have not seen this problem yet
(though I tested only on X550_X(Xeon D 12xx)'s virtual function), but we
do rearm in case TX device timeout happen.
- ixv(4): Call callout_stop() earlier in ixv_stop() like ixgbe_stop().
- KNF.

Use ixgbe_eitr_write() when writing the EITR for the link interrupt like
queue's EITR to write the register safely. This change is not relatively
so important than queue's EITR because link's EITR is written in if_init().

Remove unused structure entries. No functional change.

Fix panic when "sysctl -w hw.ixg0.txrx_workqueue=[01]" while there is traffic.

The operation is not supported, however causing panic is problem.

Read sc_if_flags after taking core lock. Same as if_wm.c rev. 1.418.

Sync with FreeBSD r331224 except ixv_if_update_admin_status()'s change:
- Fix length of reading buffer when the command is Read Flash in
ixgbe_host_interface_command().
- Add missing start_hw() call in ixv_init_locked(). This is not a real bug
because start_hw just set hw->adapter_stopped to false and anyone refer it.
- Style change.
- Update comment.

Sync with the remaining part of FreeBSD r328265 except sfp_reinit stuff:
- Always schedule module intterrupt in ixgbe_config_link() when a device is
SFP+ based.
- Use not loop index but txr->me in ixv_initialize_{transmit,receive}_units().
It's required for VMDQ but NetBSD doesn't use it, so it's not a bug in
NetBSD.
- Simplify ixgbe_bp_wd_set(). No functional change.
- Whitespace.

Fix a bug that "ifconfig ixgN down up" forgot IFM_NONE setting.
- Don't assume autoneg == 0 is the first call of ixgbe_config_link().
Check ifm_media, too.
- Don't override autoneg_advertised in ixgbe_get_phy_id_fw() to not to
be inconsistent with if_media value.

Fix long standing bug that 82598 SFP+ panics in ixgbe_handle_mod() because
hw->mac.ops.setup_sfp is NULL. This change is a part of FreeBSD r327031.

Avoid issues caused by sending old packets at next link-up time.

This modification consists by the following two parts.
- drain packets in if_snd queue or corresponding txr->txr_interq
when link_active == false in ifp->if_start(), ifp->if_transmit(),
and deferred Tx processing
- drain packets in if_snd queue and all of txr->txr_interq's
at link-down time

ok by msaitoh@n.o.

- Add missing IFM_NONE support. If a interface support linkdown,
"ifconfig ixgN media none" drpos link. Not all interface can do link down.

Tested:
82598 AT2 (T)
92599 SF+(SFI) (X520-DA2)
X540
X550-T1
X550EM_x (X10SDV-8C-TLN4F)
X550EM_a (A2SDi-H-TP4F port 0, 1 (T))

Doesn't work:
X550EM_a (A2SDi-H-TP4F port 2, 3 (SFP+ (KR)))
X550EM_a (MA10-ST0 port 2, 3 (SFP+ (SFI)))
(Denverton SFP+ can't force link down because SFP+'s TX_DISABLE pin is
pull down. Is there a way to shutdown SFP+ cage's power?)
Not tested:
82598 fiber.

- Change some functions static.

Don't write EIMC directly. It is required to manage with struct ix_queue status.

XXX pullup-8

Fix the problem between eitr and link_speed.

In ixgbe_msix_que(), que->eitr_setting is limited to IXGBE_MIN_RSC_EITR_10G1G
when link_speed is 1Gbps or 10Gbps. However, que->eitr_setting is set to EITR
register in the *next* Tx/Rx interrupt. If link_speed changes from 100Mbps to
1Gbps ro 10Gbps, que->eitr_setting which is not limited can be set to EITR
register, that is, the problem fixed by ixgbe.c:r1.124 can occur in this case.

To fix this case, que->eitr_setting should be clear when link_speed is changed
or link state is changed.

Furthermore, expand the variants used for AIM (txr->bytes, txr->packets,
rxr->bytes and rxr->packets) from u32 to u64 to avoid wraparound which causes
que->eitr_setting calculation mistake.

XXX pullup-8

- Add link related softint's counter.
- Fix indent.

Fix race about writing adapter->link_active for ixg(4).

adapter->link_active is updated by ixgbe_update_link_status() only.
The function is called from the following four functions.
- ixgbe_media_status()
- ixgbe_local_timer1()
- ixgbe_stop()
- ixgbe_handle_link()

The functions other than ixgbe_handle_link() call ixgbe_update_link_status()
with holding IXGBE_CORE_LOCK, however ixgbe_handle_link() calls it without
holding IXGBE_CORE_LOCK. That can cause race. So, add IXGBE_CORE_LOCK to
ixgbe_handle_link().

Tested by msaitoh@n.o and me.

Add some changes from ix-3.2.17.tar.gz and r328265. Not fully synchronized.
Some others (e.g. sfp cage interrupt and bypass adapter stuff) will be merged
later:
- Initialize firmware command buffer correctly in ixgbe_read_ee_hostif_X550()
and ixgbe_read_ee_hostif_buffer_X550(). These functions are used when reading
NVM.
- Fix a bug that ixgbe_mng_present() misunderstand management capability on
X550 and newer on some environment. X550 changed FWSM bit definition.
See X540 document and X550's document and compare them.
- Fix checksum calculation in ixgbe_set_fw_drv_ver_generic(). This function is
not used in NetBSD.
- Add some unused funtions.
- Whitespace fix.

Make some event counters MP safe. Now all of the event counters are
MP safe.

Reduce duplicated code which schedule deferred packet processing. No functional change.

Fix INTx/MSI handler did not schedule workqueue. Pointed out by msaitoh@n.o.

Fix another poll mode assumption breaking. Implemented by msaitoh@n.o, I just commit by proxy.

ixgbe_rearm_queues() writes EICS register(s). 82599, X540 and X550
specifications say "Following a write of 1b to any bit in the EICS register
(interrupt cause set), its corresponding bit in the EIMS register is auto
set as well enabling its interrupt." in "Extended Interrupt Auto Mask Enable
(EIAM) Register" section. That is, ixgbe_rearm_queues() causes interrupts
regardless of the status managed by ixgbe_enable_queue()/ixgbe_disable_queue().
That can break poll mode assumption.

In fact, the problem occurs in the following situation
- CPU#A has high load traffic, in contrast, CPU#B has not so high load traffic
- CPU#A is occurred interrupt by its NIC queue
- CPU#A calls ixgbe_disable_queue() in interrupt handler(ixgbe_msix_que())
- CPU#A kick softint handler(ixgbe_handle_que())
- CPU#A begins softint
- CPU#A's NIC queue is set que->txr->busy flag
- With some reason, CPU#A can do ixg interrupt handler
E.g. when one of CPU#A's softnet handlers sleeps, ipl is lowered
- CPU#B starts callout
- CPU#B calls ixgbe_local_timer1()
- CPU#B writes EICS bit corresponding CPU#A's NIC queue bit
- CPU#A's NIC queue causes interrupt whie CPU#A is running in poll mode
- CPU#A calls ixgbe_disable_queue() in interrupt handler *again*
- CPU#A has done polling, and then CPU#A calls ixgbe_enable_queue() *once*
- CPU#A's NIC queue interrupt is disabled until ixg is detached as
ixgbe_disable_queue() is called twice though ixgbe_disable_queue() is
called once only

NOTE:
82598 does not say so, but it is treated in the same way because of no harm.

By the way, we will refactor ixgbe_local_timer(watchdog processing) later.

XXX pullup-8

Don't increment que->req.ev_count in MSI-X interrupt because it's not
reschedule.

- Make "Handled queue in softint" and "Requeued in softint" evcnt(9) per queue
and count them correctly.
- Remove #if 0'ed code.

branches: 1.128.2;
ixg(4) supports workqueue poll mode, but not enabled by default yet. (that is, the default behavior is *not* changed)

At the time of high load near the wire rate, the turnaround time
of update/delete processing such as "ifconfig ixg0 inet XXX" or
"ifconfig ixg0 delete" is very long. The main reason is CPU
starvation caused by ixg(4)'s softint poll mode. ixg(4) uses
workqueue poll mode instead of softint poll mode, so that this
problem will be fix.

This change may cause performance issues, so it is not enabled
by default yet. Although there are that issues, if you want to use
workqueue poll mode, do "sysctl -w hw.ixgXX.txrx_workqueue=1" while
there is no traffic on the ixgXX.

ok by msaitoh@n.o.

Fix poll mode assumption breaking.

ixgbe_{enable,disable}_intr() forcibly enable/disable all interrupts
regardless of current state. That can break poll mode assumption,
that is, queue interrupts must not occur while polling Tx/Rx rings.

E.g. "ifconfig ixg0 delete && ifconfig ixg0 192.168.0.1" on heavy
load traffic can causes this issue.

This fix may have 1% or 2% performance impact at short packets.

XXX
ixgbe_rearm_queues() which is called only via watchdog can also break
this poll mode assumption because writing EICS casues interrupts
immediately when interrupt auto mask enabled.
We will fix it with other issues about watchdog later.

ok by msaitoh@n.o.

Fix a potential bug that TX/RX might stall when TX rate limit reached.
This change is almost the same as the RX rate limit bug fix in ixgbe.c
rev. 1.121. I've never got any stall, but this must be a bug.

Improve a comment about reading EICS register defined write-only by spec.

It seems that is workaround for silicon errata.

ok by msaitoh@n.o.

- Fix a bug that RX may stall on heavy load on ixg(4) derived from FreeBSD's
AIM (Auto Interrupt Moderation) bug.
When I use a machine as a NFS client, sometimes one of queue pairs doesn't
get any interrupt other than every second tick via ixgbe_local_timer1().
When the problem occured, the queue pair's hw.ixgM.qN.interrupt_rate is
always 500000. When this problem occuring, set hw.ixgM.qN.interrupt_rate lower
than 166667 recover from stall. i.e.:

sysctl -w hw.ixgM.qN.interrupt_rate=166667 (don't revocer)
sysctl -w hw.ixgM.qN.interrupt_rate=166666 (recover)

Relatios between the interrupt_rate and EICR's ITR_INTERVAL field is as
follows:

int_rate | EICR[11:0] | interval in us | recover |
|(ITR_INTERVAL)| (10G and 1G) | |
---------+--------------+----------------+---------+
500000 | 0x008(0) | 2 | not |
166667 | 0x010(1) | 4 | not |
166666 | 0x018(2) | 6 | recover |

The reason why int_rate becomes 500000 is that xgbe_tx_eof() doesn't
increment rxr->packets(*1). Even if we fix rxr->packets' bug, interrupt_rate
might become greater than 166666 and it might cause stall.

While reading datasheets, knakahara noticed a section titled with "ITR
Affect on RSC Functionality". It says "When RSC is enabled on specific RX
queues, the associated ITR interval with these queus must be enabled and must
be larger (in time uints) than RSC delay". Currently, RSC_DELAY field in the
GPIE register is 0 and it means 4us for 10G and 1G. The greater ITR_INTERVAL
value of 4us is 6us == 166666. Yes, BINGO!

This description is noted in 82599 and newer datasheets and not in 82598
datasheet. I don't know if 82598 has this limitation but, I apply this
limitation all of chips.

(*1) Note that this bug is going to be fixed in the next commit to distinct
between two different bugs.

- The bitfield of EITR register is different between 82598 and others.
Only ixgbe_msix_que() taken care of it. Make new function ixgbe_eitr_write()
and use it in all of functions which modify ITR_INTERVAL.

XXX pullup-8

Fix a bug that the first call of ifflags_cb() causes linkdown. The first
call of ix(gbe|v)_ifflags_cb() refered uninitialized adapter->if_flags.
adapter->if_flags should be initialized in the end of xxx_init().

XXX pullup-[678] (ixgbe)
XXX pullup-8 (ixv)

Apply RSS utility to ixg(4) and ixv(4).

ok by msaitoh@n.o.

Fix a bug that RX may stall on heavy load on ixg(4). ixgbe_rxeof() has loop
limit and the function returns true if a packet are still in the RX ring.
ixgbe_handle_que() didn't check the return value. Check the return vaule
and issue a softint.

This bug is derived from FreeBSD and ixv(4) has no this bug.

XXX pullup-8

CID-1427719: Integer handling issues (BAD_SHIFT). Print bus/slot info
correctly on 82599_SFP_SF_QP(DID 0x154a) and 82599_QSFP_SF_QP(DID 0x1558).

Fallback from MSI-X to MSI or INTx if MSI-X setup failed.

- If MSI can't be used on some environment, fallback to INTx correctly.
- Use single vector MSI when number of CPU is 1 to save interrupt slot.

Fix panic when only link interrupt can't be established.

Don't panic when resource shortage occured. Fixes PR#52820 reported by
kardel@:
- Don't use if_free() because ixgbe(4) don't use if_alloc().
- Move location of {ixgbe,ixv}_setup_interface() call at a position that
any error don't occur. One of the reason is that it should be. Another
reason is that it's hard to call ether_ifdetach() and if_detach() when
cold == 1 (because of pserialize_perform, xc_wait, timing of domaininit
and maybe more).

Sync with FreeBSD's r326022. All of the following changes have no influence
to netbsd:
- Check ETHERCAP_VLAN_HWTAGGING in ixgbe_setup_vlan_hw_support(). This change
has no influence to netbsd because it's enabled by default and NetBSD has
no API to disable it.
- Fix for netmap module.
- Remove never defined UDP_IPV4_EX
- Add SPDX-License-Identifier

On X540, print PHY FW Revision with %u.%x. 0x4030 will be printed
as "Revision 4.3 ID 0x0"

Fix a bug that bypass adapter's sysctls aren't set. Tested with non-genuine
X550-T2 bypass adapter:
- Call ixgbe_sysctl_instance() in ixgbe_bypass_init() to get sysctl's top node
correctly.
- ixgbe_init_device_features() refers adapter->hw.bus.func for bypass adapter.
Call set_lan_id() to set adapter->hw.bus.func before calling
ixgbe_init_device_features(). Without this, bypass sysctl's are added to
both the first and second port.
- Initalize node.sysctl_data before calling sysctl_lookup() to read correct
value.

Unify IFEF_*_MPSAFE into IFEF_MPSAFE

There are already two flags for if_output and if_start, however, it seems such
MPSAFE flags are eventually needed for all if_XXX operations. Having discrete
flags for each operation is wasteful of if_extflags bits. So let's unify
the flags into one: IFEF_MPSAFE.

Fortunately IFEF_*_MPSAFE flags have never been included in any releases, so
we can change them without breaking backward compatibility of the releases
(though the kernel version of -current should be bumped).

Note that if an interface have both MP-safe and non-MP-safe operations at a
time, we have to set the IFEF_MPSAFE flag and let callees of non-MP-safe
opeartions take the kernel lock.

Proposed on tech-kern@ and tech-net@

Remove obsoleted comment. This comment should have been removed in rev. 1.97.

On device which has SFP(+) cage and a module is inserted, hw->phy.id is not
MII PHY id but SFF 8024 ID. So checking hw->phy.id with 0 doesn't work.
Print PHY ID only for copper PHY.

- Handle hw.ixgN.advertise_speed correctly.
- Fix hw.ixgN.advertise_speed sysctl's description.

- Remove all half duplex setting.
- Remove special handling of IXGBE_DEV_ID_82598AT's 1000BaseT. The PHY is
compliant with clause 45, so the media is added in the beginning of
ixgbe_add_media_types().

- For X550 and newer, print NVM Image Version with %u.%02x (e.g. "0.05").
- Fix value check of OEM_NVM_IMAGE_VER.

If if_initialize() failed in the attach function, free resources and return.

Protect ec_multi* with mutex like other MP safe Ethernet drivers.
See if_ether.h rev. 1.66 and related stuff:
http://mail-index.netbsd.org/source-changes/2016/12/28/msg080289.html

Set baudrate correctly. A device which use mii(4) interface can set
if_baudrate atomatically by mii_phy_statusmsg(). ixg(4) and ixv(4) don't
use mii(4), so calling ifmedia_baudrate() directly is required.

XXX need the same same work for wm(4)'s TBI and SERDEV devices.

- Remove half-duplex setting for more than 1000Mbps in ixgbe_add_media_types()
because those medias are not in the spec.
- Add 2.5GBASE-T and 5GBASE-T support. Tested with X550-T1.

Example:
ixg0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
capabilities=fff80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx>
capabilities=fff80<TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx>
capabilities=fff80<TCP6CSUM_Tx,UDP6CSUM_Rx,UDP6CSUM_Tx,TSO6,LRO>
enabled=7ff80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx>
enabled=7ff80<TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx>
enabled=7ff80<TCP6CSUM_Tx,UDP6CSUM_Rx,UDP6CSUM_Tx,TSO6>
ec_capabilities=7<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
ec_enabled=7<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
address: a0:36:9f:b0:f5:b0
media: Ethernet 2.5GBASE-T full-duplex
status: active
supported Ethernet media:
media 10Gbase-T
media 10Gbase-T mediaopt full-duplex
media 1000baseT
media 1000baseT mediaopt full-duplex
media 100baseTX
media 100baseTX mediaopt full-duplex
media 2.5GBASE-T mediaopt full-duplex
media 5GBASE-T mediaopt full-duplex
media autoselect
inet 172.16.0.1/24 broadcast 172.16.0.255 flags 0x0
inet6 fe80::a236:9fff:feb0:f5b0%ixg0/64 flags 0x0 scopeid 0x1

Fix a bug that X550EM (Denverton) can't force 10BaseT.

Print device name in ixv_attach().

Fix 2500 KX media setting in last commit.

- Sync with FreeBSD r320688 (and up to r322865):
- Add C3000(Denvertion(-NS)) support.
- Add bypass function support for bypass adapters. Sysctls are provided.
Not tested because I have no any bypass adapter.
- ixv(4): set RSS mapping.
- Change EEE sysctl.
- ixv(4): Add TSOv6.
- ixv(4): Mailbox API 1.2 and more are implemented and comment says it
negotiate with 1.2 but it really does 1.1...
- Remove thermal test sysctl.
- Fix unknown bugs.
- Print driver feature capabilities and enable bits when verbose boot.

FreeBSD r320688 sorted a lot of functions. Merging that change by one commit
makes difficult to know what was really changed. To make change understandable,
commit only function sort and white space changes before real change.

Don't limit number of queue pair to 8. Take max TX queues and max RX queues
into account. 82598 has 32 TX queue and 64 RX queues. Others has 128 TX queues
and 128 RX queues. Number of MSI-X table entries is 64, so the max queue pairs
is 63 (64 minus the Others (e.g. link status change)).

Remove unused counters.

Print PHY ID.

Fix a bug of ixg(4)'s media setting.

Before:
ifconfig ixg0 media 100baseTX -> advertise 100Mbps only
ifconfig ixg0 media 1000baseT -> advertise 1Gbps and 1000Mbps (NG)
ifconfig ixg0 media 10Gbase-T -> advertise all (NG)
ifconfig ixg0 media auto -> advertise all

After:
ifconfig ixg0 media 100baseTX -> advertise 100Mbps only
ifconfig ixg0 media 1000baseT -> advertise 1Gbps only
ifconfig ixg0 media 10Gbase-T -> advertise 10Gbps only
ifconfig ixg0 media auto -> advertise all

Reduce diff against FreeBSD. change hw.ixgN.ts to hw.ixgN.thermal_test
(FreeBSD r282289).

Reduce diff against FreeBSD. No functional change.

Print verbose outout in ixgbe_update_link_status() correctly on 5Gbps and
2.5Gbps (NBASE-T) for X550T[12] and newer.

Use IFM_1000_KX more.

Use IFM_1000_KX and IFM_2500_KX.

branches: 1.88.2;
- Print NVM Image Version, PHY FW Revision, NVM Map version and OEM NVM
Image version
- s/TrackID/ETrackID/

Add missing {ixgbe,ixv}_stop() in the detach function. Without this change,
shutdown won't finish or panic on heay traffic.

Count Bad SFD (Start Frame Delimiter). X550 and newer have this register.

Clear all event counters when SIOCZIFDATA(ifconfig -z) is called.

Fix a bug that the driver prints "Link is up 1 Gbps" even if the link is
100Mbps. This change is the same as a part of ixv.c rev. 1.42.

Document says M[LR]FC are valid when link is up and 10Gbps, so:
- Add those values when link is up and 10Gbps
- Discard those vaules when link becomes up and the speed is 10Gbps

- Count "Total Packets Missed" by evcnt(9)
- Call evcnt_attach_dynamic()/detach() for stats->mngpdc
(Management Packets Dropped).
- Make stats->rnbc (Receive No Buffers) evcnt(9). Only 82598 has this register.
- Set RQSMR, TQSM (or TQSMR on 82598) to count qprc, qptc, qbrc, qbtc and qprdc
in each queue counter. Without this change, all queue's counts are counted in
queue 0's. Even if with this change, only MPC[0] is counted and other MPCs
are not counted.

Fix a bug that reading hw.ixgM.qN.interrupt_rate cause the interrupt
throtting rate register to 0. Check sysctl read/write correctly...

branches: 1.80.4;
Set PCI_INTR_MPSAFE, SOFTINT_MPSAFE, CALLOUT_MPSAFE and IFEF_START_MPSAFE
if NET_MPSAFE is defined.

Print mac.type and the TrackID.

Do if_snd check even if IXGBE_LEGACY_TX is used. It's used by ALTQ.

txt_si softint is used only for multiqueue (ixgbe_mq_start_locked()), so don't
schedule it in ixgbe_legacy_irq().

Remove more ix*_start_locked() from interrupt context like previous commit.
> Fix a bug that ix*_start_locked() is called in interrput context
> (ix*_msix_que). The function is called in softint(ix*_handle_que()). OK'd by
>k-nakahara.

Use percpuq.

Fix a bug that ix*_start_locked() is called in interrput context
(ix*_msix_que). The function is called in softint(ix*_handle_que()). OK'd by
k-nakahara.

Change the Interrupt Moderation flag from global variable to per device.

- Both mbuf_defrag_failed and m_defrag_failed existed. Remove m_defrag_failed.
- rx_irq, dropped_pkts, morerx, moretx and txloops counters are uncounted, so
remove them.
- Count handleq in ixv_handle_que().
- Detach event counters correctly.
- Set some per-queue event names correctly.

Print CPU number correctly.

Call IFQ_SET_MAXLEN() even if TX multiqueue is used. ALTQ uses it.

Add queue number into interrupt name.

Set ifm_active to IFM_NONE when the physical interface's link is down.
Now ifconfig shows "media: Ethernet autoselect '(none)'".

TX multiqueue. If you want to disable it, enable IXGBE_LEGACY_TX
in ixgbe_netbsd.h

Call ixgbe_get_supported_physical_layer() before ixgbe_add_media_types() to
set new media types correctly.

Remove debug printf()s.

Fix SFP+ insersion or replacement stuff.

Reduce diff against FreeBSD. No functional change.

Now we can use multiqueue. It's default on ixg(4). Not yet for ixv(4).

Fix sysctl stuff. Make the top sysctl node and keep the pointer.

Use 64bit DMA tag. If not, a lot of bounce buffer is allocated.
Fixes PR#49968 reported by Hauke.

XXX pullup-[67].

branches: 1.59.2;
Fix INIT_DEBUGOUT() messages.

Fix a bug that hw.ixgN.wol_enable=0 doesn't work correctly. Clear the WUC
and WUFC register if wol_enable is 0. ixgbe_setup_low_power_mode() calls
ixgbe_stop() and it resets the chip. The reset change WUC and WUFC registers to
the default, so clearing WUC and WUFC should be done after ixgbe_stop().

Allow 0 as the defalut of hw.ixgN.advertise_speed sysctl.

Fix flow control setting sysctl. I don't know why disabling autonego when
the flow control parameter is changed. Now the ixgN.fc sysctl works as
expected.

Add missing IFM_FDX.

Remove extra newline in debug printfs. ixgbe's *DEBUGOUT* macros automatically
add newline.

- Fix wol_enable, fc and wufc sysctls to work correctly.
- Fix RCS Id.

Update ixg(4) and ixv(4) up to FreeBSD r303890:
- Configure ixgbe phy & gbic power.
Setup phy and gbic power as per Linux 4.3.13 driver.
This fixes link not detected on X540-AT2 after booting to Linux which
turns the phy power off on detach. (FreeBSD r295093)
- Fixup DA cable detection routine. (FreeBSD r303032)
- Some minor changes

Update ixg(4) and ixv(4) up to FreeBSD r294578:
- Fixup SFP module insertion on the 82599 when insertion happens after
the system is booted and running. Add PHY detection logic to
ixgbe_handle_mod() and add locking to ixgbe_handle_msf() as well.
FreeBSD r293334.
- Fix ix advertise value after media change. When ifconfig sets media then the
values displayed by the advertise_speed value are invalidated.
Fix this by setting the bits correctly including setting advertise to 0 for
media = auto. FreeBSD r294578.
- Some others (e.g. LRO(not used by NetBSD)).

Fix ix{gbe,v}_set_sysctlvalue().

- Add missing pmf_device_deregister() in ixgbe_detach()
- Remove extra newline from an sysctl description.
- Fix dmesg output when the number of PCIe lane can't get.

Update ixg(4) and ixv(4) up to FreeBSD r292674:
- Add X552 SFP+ and X550T single port.
- Bug fixes.

Update ixg(4) and ixv(4) up to FreeBSD r289238:
- Add support for sysctl knobs to live tune the per interrupt rx/tx packet
processing limits in ixgbe(4)
- Some others (netmap, etc.)

Update ixg(4) and ixv(4) up to FreeBSD r285590:
- Fix igxbe SRIOV VF (if_ixv) initialization bugs. The MAC address for
an if_ixv instance can now set at creation time, and the receive ring
tail pointer is correctly initialized (previously, things still worked
because the receive ring tail pointer was being fixed up as a side
effect of other activity).

Update ixg(4) and ixv(4) up to FreeBSD r283881:
- SRIOV support (not enabled because NetBSD doesn't support it).

Update ixg(4) and ixv(4) up to FreeBSD r282299:
- Add support for X55x.
- ADD EEE support (not tested).
- Add WOL support (not tested).
- Add suspend/resume support (not testd).
- Add LPLU support (not tested).
- Add DMA Coalescing (note verified).
- Not tested well for sysctls.
- Fix ixgbe_set_advertise() a bit. At least, FreeBSD r294578 is required
to work hw.ixg0.advertise_speed sysctl correctly.

Update ixg(4) and ixv(4) up to FreeBSD r280197:
- Add support for 82599_QSFP_SF_QP and X540T1.
- Add partial support for X55x. It's required to sync with FreeBSD r282299
to support X55x.
- Bugfixes.

FreeBSD r280182 made new file ix_txrx.c and moved ixgbe.c and ixv's common
code into it. Before sync with whole of them, just move ixgbe.c and ixv.c's
common code into ix_txrx.c from ixgbe.c. In this commit, only ixgbe.c is split
into the device dependent part and the common part. ixv.c isn't change to make
this commit no functional change. To use ixv.c with ix_txrx.c, it's required
to modify the common part's API and functions themselves.

This commit is done to make the next change easy to understand.

Remove #ifdef NETBSD_MSI_OR_MSIX to be simple. ixv(4) isn't attached if
__HAVE_PCI_MSI_MSIX isn't set because MSI-X must be used.

provide intr xname

branches: 1.39.2;
pci_intr_type() is required pci_chipset_tag_t argument by other than x86.

pointed out by nonaka@n.o.

Introduce m_set_rcvif and m_reset_rcvif

The API is used to set (or reset) a received interface of a mbuf.
They are counterpart of m_get_rcvif, which will come in another
commit, hide internal of rcvif operation, and reduce the diff of
the upcoming change.

No functional change.

Introduce softint-based if_input

This change intends to run the whole network stack in softint context
(or normal LWP), not hardware interrupt context. Note that the work is
still incomplete by this change; to that end, we also have to softint-ify
if_link_state_change (and bpf) which can still run in hardware interrupt.

This change softint-ifies at ifp->if_input that is called from
each device driver (and ieee80211_input) to ensure Layer 2 runs
in softint (e.g., ether_input and bridge_input). To this end,
we provide a framework (called percpuq) that utlizes softint(9)
and percpu ifqueues. With this patch, rxintr of most drivers just
queues received packets and schedules a softint, and the softint
dequeues packets and does rest packet processing.

To minimize changes to each driver, percpuq is allocated in struct
ifnet for now and that is initialized by default (in if_attach).
We probably have to move percpuq to softc of each driver, but it's
future work. At this point, only wm(4) has percpuq in its softc
as a reference implementation.

Additional information including performance numbers can be found
in the thread at tech-kern@ and tech-net@:
http://mail-index.netbsd.org/tech-kern/2016/01/14/msg019997.html

Acknowledgment: riastradh@ greatly helped this work.
Thank you very much!

Add kernel code to support intrctl(8).

Reduce diff against FreeBSD r280181.

- Add MSI/MSI-X support. The multiqueue function is not supported yet.
- Make ixv.c compilable. _NOT_TESTED_YET_

Sync ixg(4) up to FreeBSD r279805 (or r280181) which include some bugfixes.
TODO:
- Merge r280182 and newer. It's required to support X55x.
- MSI/MSI-X support.

Fix error path in ixgbe_attach() and ixgbe_detatch() to prevent panic.

Add missing evcnt_detach(&rxr->rx_copies) in ixgbe_detach().

remove unreachable code.

Fixes link error without vlan. PR#49879.

Sync ixg(4) up to FreeBSD r250108:
- Cleanup some unused counters and some unused code.
- Improve performance.
- Fix flow control - don't override user value on re-init
- Fix to make 1G optics work correctly
- Change to interrupt enabling - some bits were incorrect
for certain hardware.
- Certain stats fixes, remove a duplicate increment of
ierror, thanks to Scott Long for pointing these out.
- Fix the setting of RX which related to multicast.
- Some netmap related fixes.

Fix a bug that ifconfig -z (SOICZIFDATA) doesn't work.

Sync ixg(4) up to FreeBSD r243716:
- A lot of bugfixes. Some of them are realted to multi queue and those
have not affected in NetBSD because we have not used it yet.
- Show 1000Base-SX correctly.
- Fix if_baudrate from 1G to 10G.
- Improve performance.

Update our ixg(4) driver up to FreeBSD r238149:
- Add TSO6 support.
- The max size in dma tag is changed from 65535 to 262140 (IXGBE_TSO_SIZE).
The value is the same as other *BSDs. The change might cause a address
space shortage (ixgbe_dmamap_create() might fail) on some machines.
- Fix a lot of bugs.
- Improve performance.

Synchronize our ixg(4) driver up to FreeBSD r230775:
- Add X540 support.
- Add 100BaseTX support.
- Fix a lot of bugs.
- Improve performance.

Fix a bug that vlan setting may not apply to the parent interface correctly.

Sync ixg(4) up to FreeBSD r230572.

- Fix in the interrupt handler to make sure the stack TX queue is processed.
(FreeBSD r222588)
- The maximum read size of incoming packets is done in 1024-byte increments.
The current code was rounding down the maximum frame size instead of
routing up, resulting in a read size of 1024 bytes, in the non-jumbo
frame case, and splitting the packets across multiple mbufs.
(FreeBSD r225045)
- Consequently the above problem exposed another issue, which is when
packets were splitted across multiple mbufs, and all of the mbufs in the
chain have the M_PKTHDR flag set. (FreeBSD r225045)
- Use the correct constant for conversion between interrupt rate
and EITR values (the previous values were off by a factor of 2)
(FreeBSD r230572)
- Make dev.ix.N.queueM.interrupt_rate a RW sysctl variable. Changing
individual values affects the queue immediately, and propagates to all
interfaces at the next reinit. (FreeBSD r230572)
- Add dev.ix.N.queueM.irqs rdonly sysctl, to export the actual interrupt
counts. (FreeBSD r230572)
- Some netmap related changes.

Add 82599EN_SFP and 82599_SFP_SF_QP. Not tested.

Change macro name IXGBE_DEV_ID_82599_SFP_DELL to IXGBE_DEV_ID_82599_SFP_SF2.
Same as OpenBSD and Linux. No binary change.

disable preemption while scheduling the softint to configure the link.

Fix mutex related problem reported by Uwe Toenjes in PR#49328:
- Revert ixgbe_netbsd.c rev. 1.2
- make CORE_LOCK adaptive
- Release spin lock while reinitializing the jumb buffer structure.

Remove extra IXGBE_TX_LOCK()/IXGBE_TX_UNLOCK() from
ixgbe_free_transmit_structures(). This function is called in the end of
the detach function. if_stop was called and almost all resources were
freed, so it's not required to block the TX stuff.

Add missing IXGBE_RX_LOCK_DESTROY() for the detach path.

Fix a legacy interrupt problem. If the INTx line was shared with another
device, the ixgbe_legacy_irq() enabled the interrupt even if the interface
was down. Check the interface state and call functions appropriately.

branches: 1.14.2; 1.14.4;
only attach/detach stats->mngptc once.
remove a low-value printf().

CID/1008347, CID/1008348, CID/1008349: overflow before widen
CID/1203198, CID/1203199: Uninitialized scalar variable

Add __diagused.

Mark ixgbe_disable_queue as unused.

Add dell's flavor (from uwe, in current-users@)

branches: 1.9.2;
Make this compile

- Fix pci_intr_string usage
- Remove ununsed
- define variables with their usage

kern/48690: ixg* driver doesn't compile

Untested.

make pci_intr_string and eisa_intr_string take a buffer and a length
instead of relying in local static storage.

Ensure that the top level sysctl nodes (kern, vfs, net, ...) exist before
the sysctl link sets are processed, and remove redundancy.

Shaves >13kB off of an amd64 GENERIC, not to mention >1k duplicate
lines of code.

Remove a few noisy debug printf.

branches: 1.5.2; 1.5.4;
Missed one of the (void *) casts on the args to sysctl_createv().

Add explicit (void *) casts for when sysctl_createv() is passed
CTLTYPE_INT but the address of a structure (and a function).

Add some pre-processor magic to verify that the type of the data item
passed to sysctl_createv() actually matches the declared type for
the item itself.
In the places where the caller specifies a function and a structure
address (typically the 'softc') an explicit (void *) cast is now needed.
Fixes bugs in sys/dev/acpi/asus_acpi.c sys/dev/bluetooth/bcsp.c
sys/kern/vfs_bio.c sys/miscfs/syncfs/sync_subr.c and setting
AcpiGbl_EnableAmlDebugObject.
(mostly passing the address of a uint64_t when typed as CTLTYPE_INT).
I've test built quite a few kernels, but there may be some unfixed MD
fallout. Most likely passing &char[] to char *.
Also add CTLFLAG_UNSIGNED for unsiged decimals - not set yet.

branches: 1.2.4;
First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>. This change includes
the following:

An initial cleanup and minor reorganization of the entropy pool
code in sys/dev/rnd.c and sys/dev/rndpool.c. Several bugs are
fixed. Some effort is made to accumulate entropy more quickly at
boot time.

A generic interface, "rndsink", is added, for stream generators to
request that they be re-keyed with good quality entropy from the pool
as soon as it is available.

The arc4random()/arc4randbytes() implementation in libkern is
adjusted to use the rndsink interface for rekeying, which helps
address the problem of low-quality keys at boot time.

An implementation of the FIPS 140-2 statistical tests for random
number generator quality is provided (libkern/rngtest.c). This
is based on Greg Rose's implementation from Qualcomm.

A new random stream generator, nist_ctr_drbg, is provided. It is
based on an implementation of the NIST SP800-90 CTR_DRBG by
Henric Jungheim. This generator users AES in a modified counter
mode to generate a backtracking-resistant random stream.

An abstraction layer, "cprng", is provided for in-kernel consumers
of randomness. The arc4random/arc4randbytes API is deprecated for
in-kernel use. It is replaced by "cprng_strong". The current
cprng_fast implementation wraps the existing arc4random
implementation. The current cprng_strong implementation wraps the
new CTR_DRBG implementation. Both interfaces are rekeyed from
the entropy pool automatically at intervals justifiable from best
current cryptographic practice.

In some quick tests, cprng_fast() is about the same speed as
the old arc4randbytes(), and cprng_strong() is about 20% faster
than rnd_extract_data(). Performance is expected to improve.

The AES code in src/crypto/rijndael is no longer an optional
kernel component, as it is required by cprng_strong, which is
not an optional kernel component.

The entropy pool output is subjected to the rngtest tests at
startup time; if it fails, the system will reboot. There is
approximately a 3/10000 chance of a false positive from these
tests. Entropy pool _input_ from hardware random numbers is
subjected to the rngtest tests at attach time, as well as the
FIPS continuous-output test, to detect bad or stuck hardware
RNGs; if any are detected, they are detached, but the system
continues to run.

A problem with rndctl(8) is fixed -- datastructures with
pointers in arrays are no longer passed to userspace (this
was not a security problem, but rather a major issue for
compat32). A new kernel will require a new rndctl.

The sysctl kern.arandom() and kern.urandom() nodes are hooked
up to the new generators, but the /dev/*random pseudodevices
are not, yet.

Manual pages for the new kernel interfaces are forthcoming.

branches: 1.1.2;
Add sources for ixgbe(4), a Intel 82599 10-gigabit ethernet driver
ported from FreeBSD.

Fix typos in comments, mainly s/calcurate/calculate/.

s/trafic/traffic/ and s/Crrently/Currently/ in comments.

branches: 1.98.2;
ixgbe: Use #ifdef RSC

This feature (hardware receive side coalescing) has been disabled all along,
so enclose the code with #ifdef RSC.

ixgbe: Change "me" from 32bit to 8bit because the max is 128.

This commit doesn't change the real size of ix_queue, tx_ring and rx_ring
because of the alignment.

ixgbe: Use #ifdef LRO more to reduce the size of struct rx_ring.

ixgbe: Remove unused to reduce the size of struct rx_ring.

ixgbe: Use #ifdef IXGBE_FDIR more

Don't include the Flow Director related members to reduce the size of
struct tx_ring. On amd64 and aarch64, the real size is not changed
because of the alignment.

ixgbe: Don't override the {ixgbe,ixv}_max_interrupt_rate global variable.

Fix a bug that changing hw.ix[gv]X.qY.interrupt_rate would change all
devices all queues default interrupt rate.

ixgbe: Cleanup. No fucntional change.

Remove unused code, fix whitespace and modify comment to reduce against
FreeBSD(mainly from ix-3.3.35).

ixgbe: Simplify definitions. No functional change.

Both DEFAULT_{TX,RX}D and PERFORM_{TX,RX}D are 2048. Use DEFAULT_{TX,RX}D.
Same as FreeBSD.

ixg(4): Remove unused and old function. No functional change.

- From FreeBSD's ix-3.3.32.