Add a COMMENT describing what each component roughly does.

"make describe" prints the comment.

Requested/inspired by Vincent Schwarzer on rumpkernel-users

branches: 1.9.6;
Rename rumpcomp_user.c -> virtif_user.c
Convert to new-style RUMPCOMP_USER_SRCS in Makefile.

Separate common bits for all interfaces implementing virtif hypercalls
from Makefile to Makefile.virtif. The latter is meant to be .included.

Determine the interface name and the names of exported symbols based
on VIRTIF_BASE. This allows using various different packet-shoveling
backends in the same rump kernel.

branches: 1.6.6;
Rerevert to preprevious now that this builds with build.sh

Guess the RUMPCOMP_USER stuff wasn't ready to be in the NetBSD tree yet,
so revert previous commits to fix build. I'll look at rereverting when
toolchain/47644 is fixed and clean buildtests are again possible.

Push the bits accessing the networking backend into hypercalls.
This allows a few highly desirable things to happen:

1) Linux support, which is included in this commit (thanks to wanq for
sending me the magic bits for accessing /dev/net/tun)
2) Reusing the kernel-side of the interface for devices such as
hardware packet processors which interact directly with user vm spaces
(not included in the commit).

branches: 1.3.12; 1.3.22;
Use a bunch of weak symbols to determine which network components
are present. This works in userspace as opposed relying in link
sets, which fail miserably. Later, when the networking stack
becomes modularized, we can move to a dynamic scheme like with file
systems.

Also, this change allows us to do proper autoconfig, namely attach
the loopback interface iff it is present.

branches: 1.2.2; 1.2.10; 1.2.12; 1.2.16;
kill default DBG=-g

branches: 1.1.2;
Implement a virtual userspace networking interface. This is required
for any sensible operation when running the entire TCP/IP stack in
userspace (as opposed to libsockin which uses host kernel networking
to provide PF_INET).

While this basically works (although it is quite barebones), it
depends on some bit of cleanup in librump and is not built by
default yet.

branches: 1.4.4; 1.4.6; 1.4.10;
No reason to attach if_virt as a RUMP_COMPONENT(), simply use MODULE().

remove unnecessary header inclusions

component.c -> virtif_component.c, just to decrease risk of conflicts.
No runtime change.

Separate common bits for all interfaces implementing virtif hypercalls
from Makefile to Makefile.virtif. The latter is meant to be .included.

component.c -> virtif_component.c, just to decrease risk of conflicts.
No runtime change.

Determine the interface name and the names of exported symbols based
on VIRTIF_BASE. This allows using various different packet-shoveling
backends in the same rump kernel.

branches: 1.3.8; 1.3.18; 1.3.22;
Make virtif a cloner so that it can be brough to life also with
"ifconfig create". As previously, virt<n> interfaces with the
host's /dev/tap<n> (I guess it could be made explicit with
"ifconfig media", but leave it this way for now).

branches: 1.2.2;
Introduce RUMP_COMPONENT. It behaves mostly like a simplified
module which is linked into the kernel and cannot be unloaded.
The main purpose is to get the proper constructors run and create
any /dev nodes necessary for said component. Once more of the
kernel (e.g. networking stack and device drivers) are converted to
MODULE and devfs pops up from somewhere, rump components can be
retired.

branches: 1.1.2; 1.1.4;
Use a bunch of weak symbols to determine which network components
are present. This works in userspace as opposed relying in link
sets, which fail miserably. Later, when the networking stack
becomes modularized, we can move to a dynamic scheme like with file
systems.

Also, this change allows us to do proper autoconfig, namely attach
the loopback interface iff it is present.

if_attach and if_initialize cannot fail, don't test return value

These were originally made failable back in 2017 when if_initialize
allocated a softint in every interface for link state changes, so
that it could fail gracefully instead of panicking:

https://mail-index.NetBSD.org/source-changes/2017/10/23/msg089053.html

However, this spawned many seldom- or never-tested error branches,
which are risky to have around. And that softint in every interface
has since been replaced by a single global workqueue, because link
state changes require thread context but not low latency or high
throughput:

https://mail-index.NetBSD.org/source-changes/2020/02/06/msg113759.html

So there is no longer any reason for if_initialize to fail. (The
subroutine if_stats_init can't fail because percpu_alloc can't fail
either.)

There is a snag: the softint_establish in if_percpuq_create could
fail, potentially leading to bad consequences later on trying to use
the softint. This change doesn't introduce any new bugs because of
the snag -- if_percpuq_attach was already broken. However, the snag
can be better addressed without spawning error branches, either by
using a single softint or making softints less scarce.

(Separate commit will change the signatures of if_attach and
if_initialize to return void, scheduled to ride whatever is the next
convenient kernel bump.)

Patch and testing on amd64 and evbmips64-eb by maya@; commit message
soliloquy, and compile-testing on evbppc/i386/earmv7hf, by me.

branches: 1.58.10;
Adopt <net/if_stats.h>.

branches: 1.57.2; 1.57.8;
Implement the BPF direction filter (BIOC[GS]DIRECTION). It provides backward
compatibility with BIOC[GS]SEESENT ioctl. The userland interface is the same
as FreeBSD.

This change also fixes a bug that the direction is misunderstand on some
environment by passing the direction to bpf_mtap*() instead of checking
m->m_pkthdr.rcvif.

branches: 1.56.2;
Fix compile error.

If error occured in the attach function, free resources and return.

branches: 1.54.8;
Move bpf_mtap and if_ipackets++ on Rx of each driver to percpuq if_input

The benefits of the change are:
- We can reduce codes
- We can provide the same behavior between drivers
- Where/When if_ipackets is counted up
- Note that some drivers still update packet statistics in their own
way (periodical update)
- Moved bpf_mtap run in softint
- This makes it easy to MP-ify bpf

Proposed on tech-kern and tech-net

branches: 1.53.2;
Use curlwp_bind and curlwp_bindx instead of open-coding LP_BOUND

Introduce m_set_rcvif and m_reset_rcvif

The API is used to set (or reset) a received interface of a mbuf.
They are counterpart of m_get_rcvif, which will come in another
commit, hide internal of rcvif operation, and reduce the diff of
the upcoming change.

No functional change.

Prevent LWP migrations between CPUs during upper layer processing

This is a contract of psref(9) that is used by upper layer componenets,
e.g., bridge(4).

Introduce softint-based if_input

This change intends to run the whole network stack in softint context
(or normal LWP), not hardware interrupt context. Note that the work is
still incomplete by this change; to that end, we also have to softint-ify
if_link_state_change (and bpf) which can still run in hardware interrupt.

This change softint-ifies at ifp->if_input that is called from
each device driver (and ieee80211_input) to ensure Layer 2 runs
in softint (e.g., ether_input and bridge_input). To this end,
we provide a framework (called percpuq) that utlizes softint(9)
and percpu ifqueues. With this patch, rxintr of most drivers just
queues received packets and schedules a softint, and the softint
dequeues packets and does rest packet processing.

To minimize changes to each driver, percpuq is allocated in struct
ifnet for now and that is initialized by default (in if_attach).
We probably have to move percpuq to softc of each driver, but it's
future work. At this point, only wm(4) has percpuq in its softc
as a reference implementation.

Additional information including performance numbers can be found
in the thread at tech-kern@ and tech-net@:
http://mail-index.netbsd.org/tech-kern/2016/01/14/msg019997.html

Acknowledgment: riastradh@ greatly helped this work.
Thank you very much!

branches: 1.49.2;
Ensure that no two VIRTIF's have the same modname, therefore
allowing them to coexist in the same rump kernel.

from Alexander Guy, via github

Count packets in if_virt

use if_initname() instead of homegrown sprintf

kill sprintf

Trim alignment out of packet length before passing it up.

No reason to attach if_virt as a RUMP_COMPONENT(), simply use MODULE().

remove unnecessary header inclusions

allow VIF_DYING() implementation to protest

Rename rumpcomp_user.c -> virtif_user.c
Convert to new-style RUMPCOMP_USER_SRCS in Makefile.

When sending, only load mbufs which have a length. This is especially
useful for fragmented datagrams where the tail of the first packet is
full of nothing.

Pull in the changes from the dpdk, netmap and snabb switch repos.
There are two major changes:

1) All thread context policy is pushed down to the hypercalls.
This is meant to help performance-mongering implementations be
able to control packet scheduling better (e.g. pin down packet
reception to certain physical cores).
2) Generalize linkstr, meaning that the interface can now take an
arbitrary string which is passed to the create() hypercall. This can
be used to map backend device characteristics to the rump kernel
interface instance.

Fix typo in fix for PR/48606

PR/48606: Lloyd Parkes: Drivers not using ifp->if_input but using ether_input
directly.

Determine the interface name and the names of exported symbols based
on VIRTIF_BASE. This allows using various different packet-shoveling
backends in the same rump kernel.

fix parameter name

Adjust for rump_virtif_create() removal. Incidentally, this reduces
the number of exported symbols by one (which is useful for phase 2).

g/c dummyif, not useful these days

Allow alternate interface base names to be specified at compile time.

branches: 1.31.4;
Make hypercall calling conventions consistent: iff a hypercall can fail,
it returns an int containing the error value.

Rerevert to preprevious now that this builds with build.sh

Guess the RUMPCOMP_USER stuff wasn't ready to be in the NetBSD tree yet,
so revert previous commits to fix build. I'll look at rereverting when
toolchain/47644 is fixed and clean buildtests are again possible.

Push the bits accessing the networking backend into hypercalls.
This allows a few highly desirable things to happen:

1) Linux support, which is included in this commit (thanks to wanq for
sending me the magic bits for accessing /dev/net/tun)
2) Reusing the kernel-side of the interface for devices such as
hardware packet processors which interact directly with user vm spaces
(not included in the commit).

Do not assume that O_FOO flags have matching values in the rump kernel
and hypervisor.

branches: 1.26.8;
First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>. This change includes
the following:

An initial cleanup and minor reorganization of the entropy pool
code in sys/dev/rnd.c and sys/dev/rndpool.c. Several bugs are
fixed. Some effort is made to accumulate entropy more quickly at
boot time.

A generic interface, "rndsink", is added, for stream generators to
request that they be re-keyed with good quality entropy from the pool
as soon as it is available.

The arc4random()/arc4randbytes() implementation in libkern is
adjusted to use the rndsink interface for rekeying, which helps
address the problem of low-quality keys at boot time.

An implementation of the FIPS 140-2 statistical tests for random
number generator quality is provided (libkern/rngtest.c). This
is based on Greg Rose's implementation from Qualcomm.

A new random stream generator, nist_ctr_drbg, is provided. It is
based on an implementation of the NIST SP800-90 CTR_DRBG by
Henric Jungheim. This generator users AES in a modified counter
mode to generate a backtracking-resistant random stream.

An abstraction layer, "cprng", is provided for in-kernel consumers
of randomness. The arc4random/arc4randbytes API is deprecated for
in-kernel use. It is replaced by "cprng_strong". The current
cprng_fast implementation wraps the existing arc4random
implementation. The current cprng_strong implementation wraps the
new CTR_DRBG implementation. Both interfaces are rekeyed from
the entropy pool automatically at intervals justifiable from best
current cryptographic practice.

In some quick tests, cprng_fast() is about the same speed as
the old arc4randbytes(), and cprng_strong() is about 20% faster
than rnd_extract_data(). Performance is expected to improve.

The AES code in src/crypto/rijndael is no longer an optional
kernel component, as it is required by cprng_strong, which is
not an optional kernel component.

The entropy pool output is subjected to the rngtest tests at
startup time; if it fails, the system will reboot. There is
approximately a 3/10000 chance of a false positive from these
tests. Entropy pool _input_ from hardware random numbers is
subjected to the rngtest tests at attach time, as well as the
FIPS continuous-output test, to detect bad or stuck hardware
RNGs; if any are detected, they are detached, but the system
continues to run.

A problem with rndctl(8) is fixed -- datastructures with
pointers in arrays are no longer passed to userspace (this
was not a security problem, but rather a major issue for
compat32). A new kernel will require a new rndctl.

The sysctl kern.arandom() and kern.urandom() nodes are hooked
up to the new generators, but the /dev/*random pseudodevices
are not, yet.

Manual pages for the new kernel interfaces are forthcoming.

branches: 1.25.2;
fix a type in a printf message

Rename slightly misleading KTHREAD_JOINABLE to KTHREAD_MUSTJOIN.

fix an operator precedence error picked up by GCC 4.5.3. real bug.

Don't bother asserting: if we create the thread without KTHREAD_MPSAFE,
it's not going to be MPSAFE.

Make interface support ifconfig {down,destroy} and generally make
it a little less eager to panic.

Make virtif a cloner so that it can be brough to life also with
"ifconfig create". As previously, virt<n> interfaces with the
host's /dev/tap<n> (I guess it could be made explicit with
"ifconfig media", but leave it this way for now).

* improve diagnostic print
* deal with a tap quirk when it returns 0 bytes

Fix typo

Push the bpf_ops usage back into bpf.h. Push the common ifp->if_bpf
check into the inline functions as well the fourth argument for
bpf_attach.

branches: 1.16.2; 1.16.4;
Redefine bpf linkage through an always present op vector, i.e.
#if NBPFILTER is no longer required in the client. This change
doesn't yet add support for loading bpf as a module, since drivers
can register before bpf is attached. However, callers of bpf can
now be modularized.

Dynamically loadable bpf could probably be done fairly easily with
coordination from the stub driver and the real driver by registering
attachments in the stub before the real driver is loaded and doing
a handoff. ... and I'm not going to ponder the depths of unload
here.

Tested with i386/MONOLITHIC, modified MONOLITHIC without bpf and rump.

support bpf

"rumppriv" goes back to "rump" per internal interface naming change.

Adjust rump sources for external/internal interfaces.
No functional change.

Bump iovec lazy bum magic value to 32: nfsd likes to write mbufs
with 17 per chain (previous i'm-too-lazy-for-my-forloop value was
of course 16).

work around tap bug: if /dev/tap<n> was previously non-blocking, newly
opened fd's will also be non-blocking.
(yeayea, i'll fix the kernel some day, but I don't want to reboot
my host OS now)

Add a dummyif, which doesn't actually traffic any cargo, but since
it has no backend it can always be attached and is therefore
convenient for testing ifconfig.

Filter out ENETRESET from ether_ioctl() since we aren't interested
in multicast hugging.

branches: 1.8.2;
* make interface creation open /dev/tapn for interface n
* create "unique" enaddr
* do send in async context

Fix a silly mistake: indubitably the struct ifnet pointer given to
ether_ifattach() should point to a struct ethercom.

decouple from host struct iovec

branches: 1.5.2; 1.5.4;
__KERNEL_RCSID

branches: 1.4.2; 1.4.4;
Add a prototype to allow this to build (will be moved to a better
place later).

Make the "something random" ethernet address ETHER_IS_LOCAL
(address assignment probably needs some more work if we want to
support multiple interfaces).

hat tip to Quentin Garnier

Dear diary: an ethernet address with 0x01 set in the first octet
is an ethernet multicast address. TCP does not like ethernet
multicast addresses.

branches: 1.1.2;
Implement a virtual userspace networking interface. This is required
for any sensible operation when running the entire TCP/IP stack in
userspace (as opposed to libsockin which uses host kernel networking
to provide PF_INET).

While this basically works (although it is quite barebones), it
depends on some bit of cleanup in librump and is not built by
default yet.

branches: 1.3.4; 1.3.8;
Pull in the changes from the dpdk, netmap and snabb switch repos.
There are two major changes:

1) All thread context policy is pushed down to the hypercalls.
This is meant to help performance-mongering implementations be
able to control packet scheduling better (e.g. pin down packet
reception to certain physical cores).
2) Generalize linkstr, meaning that the interface can now take an
arbitrary string which is passed to the create() hypercall. This can
be used to map backend device characteristics to the rump kernel
interface instance.

branches: 1.2.4;
Add a note explaining why there's a homegrown __STRING, __CONCAT, etc.

Determine the interface name and the names of exported symbols based
on VIRTIF_BASE. This allows using various different packet-shoveling
backends in the same rump kernel.

Rename rumpcomp_user.c -> virtif_user.c
Convert to new-style RUMPCOMP_USER_SRCS in Makefile.

Pull in the changes from the dpdk, netmap and snabb switch repos.
There are two major changes:

1) All thread context policy is pushed down to the hypercalls.
This is meant to help performance-mongering implementations be
able to control packet scheduling better (e.g. pin down packet
reception to certain physical cores).
2) Generalize linkstr, meaning that the interface can now take an
arbitrary string which is passed to the create() hypercall. This can
be used to map backend device characteristics to the rump kernel
interface instance.

Give the hypercall interface a version number. This is for
implementations outside of the tree.

A simple (void) is apparently too easy and traditional to make
Wunused-result STFU, so let's invent something a bit more verbose to
try to achieve the desired result of "ccg xnaht I really don't care if
you think I should check the return value".

Explicitly ignore return value of writev() for sending a packet.

Otherwise you get the following with some compilers:
warning: ignoring return value of 'writev', declared with attribute warn_unused_result [-Wunused-result]

Determine the interface name and the names of exported symbols based
on VIRTIF_BASE. This allows using various different packet-shoveling
backends in the same rump kernel.

fix warnings

branches: 1.6.2; 1.6.4;
Return 0 for success instead of returning an uninitialized automatic variable.

from Gal Sagie, PR 47837

Translate error codes between the hypervisor and rump kernel.

Make hypercall calling conventions consistent: iff a hypercall can fail,
it returns an int containing the error value.

fix the build!

Fix a few minor issues found by proofreading the diff one last time.

Push the bits accessing the networking backend into hypercalls.
This allows a few highly desirable things to happen:

1) Linux support, which is included in this commit (thanks to wanq for
sending me the magic bits for accessing /dev/net/tun)
2) Reusing the kernel-side of the interface for devices such as
hardware packet processors which interact directly with user vm spaces
(not included in the commit).

Rename rumpcomp_user.c -> virtif_user.c
Convert to new-style RUMPCOMP_USER_SRCS in Makefile.

Pull in the changes from the dpdk, netmap and snabb switch repos.
There are two major changes:

1) All thread context policy is pushed down to the hypercalls.
This is meant to help performance-mongering implementations be
able to control packet scheduling better (e.g. pin down packet
reception to certain physical cores).
2) Generalize linkstr, meaning that the interface can now take an
arbitrary string which is passed to the create() hypercall. This can
be used to map backend device characteristics to the rump kernel
interface instance.

Give the hypercall interface a version number. This is for
implementations outside of the tree.

Determine the interface name and the names of exported symbols based
on VIRTIF_BASE. This allows using various different packet-shoveling
backends in the same rump kernel.

branches: 1.3.4; 1.3.6;
Make hypercall calling conventions consistent: iff a hypercall can fail,
it returns an int containing the error value.

Fix a few minor issues found by proofreading the diff one last time.

Push the bits accessing the networking backend into hypercalls.
This allows a few highly desirable things to happen:

1) Linux support, which is included in this commit (thanks to wanq for
sending me the magic bits for accessing /dev/net/tun)
2) Reusing the kernel-side of the interface for devices such as
hardware packet processors which interact directly with user vm spaces
(not included in the commit).

Remove shlib_version files and just use Makefile SHLIB_MAJOR/MINOR,
with the default provided by Makefile.rump (they're all 0.0 anyway)

branches: 1.1.2; 1.1.4; 1.1.12; 1.1.18; 1.1.46; 1.1.66;
Implement a virtual userspace networking interface. This is required
for any sensible operation when running the entire TCP/IP stack in
userspace (as opposed to libsockin which uses host kernel networking
to provide PF_INET).

While this basically works (although it is quite barebones), it
depends on some bit of cleanup in librump and is not built by
default yet.

No reason to attach if_virt as a RUMP_COMPONENT(), simply use MODULE().

component.c -> virtif_component.c, just to decrease risk of conflicts.
No runtime change.

Protect __KERNEL_RCSID. It isn't available when compile rumpkernels under e.g. Linux.

Merge the [pgoyette-compat] branch

Add missing RCSIDs.

branches: 1.3.4; 1.3.6; 1.3.10; 1.3.34; 1.3.36;
Remember to bump revision for the "int my time VIF_DYING" change yesterday.

allow VIF_DYING() implementation to protest

Rename rumpcomp_user.c -> virtif_user.c
Convert to new-style RUMPCOMP_USER_SRCS in Makefile.

branches: 1.3.4; 1.3.6; 1.3.10;
Remember to bump revision for the "int my time VIF_DYING" change yesterday.

allow VIF_DYING() implementation to protest

Rename rumpcomp_user.c -> virtif_user.c
Convert to new-style RUMPCOMP_USER_SRCS in Makefile.