remove pf42 branch's todo.

merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.1.2; 1.1.6;
file TODO was initially added on branch yamt-pf42.

branches: 1.2.2; 1.2.4;
merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.1.1; 1.1.2; 1.1.4; 1.1.6;
file if_compat.c was initially added on branch yamt-pf42.

branches: 1.2.2; 1.2.4;
merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.1.1; 1.1.2; 1.1.4; 1.1.6;
file if_compat.h was initially added on branch yamt-pf42.

Adopt <net/if_stats.h>.

branches: 1.21.2; 1.21.8;
Implement the BPF direction filter (BIOC[GS]DIRECTION). It provides backward
compatibility with BIOC[GS]SEESENT ioctl. The userland interface is the same
as FreeBSD.

This change also fixes a bug that the direction is misunderstand on some
environment by passing the direction to bpf_mtap*() instead of checking
m->m_pkthdr.rcvif.

branches: 1.20.16;
Constify rtentry of if_output

We no longer need to change rtentry below if_output.

The change makes it clear where rtentries are changed (or not)
and helps forthcoming locking (os psrefing) rtentries.

include "ioconf.h" to get the 'void <driver>attach(int count);' prototype.

branches: 1.18.18; 1.18.36;
- Make the pf and pflog driver able to detach.
- Add code for module support.

Original patch from Jared McNeill

Push the bpf_ops usage back into bpf.h. Push the common ifp->if_bpf
check into the inline functions as well the fourth argument for
bpf_attach.

branches: 1.16.2; 1.16.4;
Redefine bpf linkage through an always present op vector, i.e.
#if NBPFILTER is no longer required in the client. This change
doesn't yet add support for loading bpf as a module, since drivers
can register before bpf is attached. However, callers of bpf can
now be modularized.

Dynamically loadable bpf could probably be done fairly easily with
coordination from the stub driver and the real driver by registering
attachments in the stub before the real driver is loaded and doing
a handoff. ... and I'm not going to ponder the depths of unload
here.

Tested with i386/MONOLITHIC, modified MONOLITHIC without bpf and rump.

Remove LKM code from pf.

use M_ZERO on malloc() and remove subsequent bzero().

*** Summary ***

When a link-layer address changes (e.g., ifconfig ex0 link
02:de:ad:be:ef:02 active), send a gratuitous ARP and/or a Neighbor
Advertisement to update the network-/link-layer address bindings
on our LAN peers.

Refuse a change of ethernet address to the address 00:00:00:00:00:00
or to any multicast/broadcast address. (Thanks matt@.)

Reorder ifnet ioctl operations so that driver ioctls may inherit
the functions of their "class"---ether_ioctl(), fddi_ioctl(), et
cetera---and the class ioctls may inherit from the generic ioctl,
ifioctl_common(), but both driver- and class-ioctls may override
the generic behavior. Make network drivers share more code.

Distinguish a "factory" link-layer address from others for the
purposes of both protecting that address from deletion and computing
EUI64.

Return consistent, appropriate error codes from network drivers.

Improve readability. KNF.

*** Details ***

In if_attach(), always initialize the interface ioctl routine,
ifnet->if_ioctl, if the driver has not already initialized it.
Delete if_ioctl == NULL tests everywhere else, because it cannot
happen.

In the ioctl routines of network interfaces, inherit common ioctl
behaviors by calling either ifioctl_common() or whichever ioctl
routine is appropriate for the class of interface---e.g., ether_ioctl()
for ethernets.

Stop (ab)using SIOCSIFADDR and start to use SIOCINITIFADDR. In
the user->kernel interface, SIOCSIFADDR's argument was an ifreq,
but on the protocol->ifnet interface, SIOCSIFADDR's argument was
an ifaddr. That was confusing, and it would work against me as I
make it possible for a network interface to overload most ioctls.
On the protocol->ifnet interface, replace SIOCSIFADDR with
SIOCINITIFADDR. In ifioctl(), return EPERM if userland tries to
invoke SIOCINITIFADDR.

In ifioctl(), give the interface the first shot at handling most
interface ioctls, and give the protocol the second shot, instead
of the other way around. Finally, let compatibility code (COMPAT_OSOCK)
take a shot.

Pull device initialization out of switch statements under
SIOCINITIFADDR. For example, pull ..._init() out of any switch
statement that looks like this:

switch (...->sa_family) {
case ...:
..._init();
...
break;
...
default:
..._init();
...
break;
}

Rewrite many if-else clauses that handle all permutations of IFF_UP
and IFF_RUNNING to use a switch statement,

switch (x & (IFF_UP|IFF_RUNNING)) {
case 0:
...
break;
case IFF_RUNNING:
...
break;
case IFF_UP:
...
break;
case IFF_UP|IFF_RUNNING:
...
break;
}

unifdef lots of code containing #ifdef FreeBSD, #ifdef NetBSD, and
#ifdef SIOCSIFMTU, especially in fwip(4) and in ndis(4).

In ipw(4), remove an if_set_sadl() call that is out of place.

In nfe(4), reuse the jumbo MTU logic in ether_ioctl().

Let ethernets register a callback for setting h/w state such as
promiscuous mode and the multicast filter in accord with a change
in the if_flags: ether_set_ifflags_cb() registers a callback that
returns ENETRESET if the caller should reset the ethernet by calling
if_init(), 0 on success, != 0 on failure. Pull common code from
ex(4), gem(4), nfe(4), sip(4), tlp(4), vge(4) into ether_ioctl(),
and register if_flags callbacks for those drivers.

Return ENOTTY instead of EINVAL for inappropriate ioctls. In
zyd(4), use ENXIO instead of ENOTTY to indicate that the device is
not any longer attached.

Add to if_set_sadl() a boolean 'factory' argument that indicates
whether a link-layer address was assigned by the factory or some
other source. In a comment, recommend using the factory address
for generating an EUI64, and update in6_get_hw_ifid() to prefer a
factory address to any other link-layer address.

Add a routing message, RTM_LLINFO_UPD, that tells protocols to
update the binding of network-layer addresses to link-layer addresses.
Implement this message in IPv4 and IPv6 by sending a gratuitous
ARP or a neighbor advertisement, respectively. Generate RTM_LLINFO_UPD
messages on a change of an interface's link-layer address.

In ether_ioctl(), do not let SIOCALIFADDR set a link-layer address
that is broadcast/multicast or equal to 00:00:00:00:00:00.

Make ether_ioctl() call ifioctl_common() to handle ioctls that it
does not understand.

In gif(4), initialize if_softc and use it, instead of assuming that
the gif_softc and ifp overlap.

Let ifioctl_common() handle SIOCGIFADDR.

Sprinkle rtcache_invariants(), which checks on DIAGNOSTIC kernels
that certain invariants on a struct route are satisfied.

In agr(4), rewrite agr_ioctl_filter() to be a bit more explicit
about the ioctls that we do not allow on an agr(4) member interface.

bzero -> memset. Delete unnecessary casts to void *. Use
sockaddr_in_init() and sockaddr_in6_init(). Compare pointers with
NULL instead of "testing truth". Replace some instances of (type
*)0 with NULL. Change some K&R prototypes to ANSI C, and join
lines.

branches: 1.12.2; 1.12.4;
merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.11.8; 1.11.10; 1.11.12; 1.11.14; 1.11.16;
use __KERNEL_RCSID()

branches: 1.10.16; 1.10.24; 1.10.26; 1.10.28;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

KNF: de-__P, bzero -> memset, bcmp -> memcmp. Remove extraneous
parentheses in return statements.

Cosmetic: don't open-code TAILQ_FOREACH().

Cosmetic: change types of variables to avoid oodles of casts: in
in6_src.c, avoid casts by changing several route_in6 pointers
to struct route pointers. Remove unnecessary casts to caddr_t
elsewhere.

Pave the way for eliminating address family-specific route caches:
soon, struct route will not embed a sockaddr, but it will hold
a reference to an external sockaddr, instead. We will set the
destination sockaddr using rtcache_setdst(). (I created a stub
for it, but it isn't used anywhere, yet.) rtcache_free() will
free the sockaddr. I have extracted from rtcache_free() a helper
subroutine, rtcache_clear(). rtcache_clear() will "forget" a
cached route, but it will not forget the destination by releasing
the sockaddr. I use rtcache_clear() instead of rtcache_free()
in rtcache_update(), because rtcache_update() is not supposed
to forget the destination.

Constify:

1 Introduce const accessor for route->ro_dst, rtcache_getdst().

2 Constify the 'dst' argument to ifnet->if_output(). This
led me to constify a lot of code called by output routines.

3 Constify the sockaddr argument to protosw->pr_ctlinput. This
led me to constify a lot of code called by ctlinput routines.

4 Introduce const macros for converting from a generic sockaddr
to family-specific sockaddrs, e.g., sockaddr_in: satocsin6,
satocsin, et cetera.

branches: 1.8.4;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

branches: 1.6.20; 1.6.22;
merge ktrace-lwp.

branches: 1.5.12;
resolve conflicts. (pf from OpenBSD 3.6, kernel part)

pflog_packet: use bpf_mtap2().
(our bpf_mtap() is more "strict" about mbufs
than openbsd's one is. eg. M_PKTHDR should be set properly.)

branches: 1.3.2;
make PF lkm working. from Peter Postma and Joel Wilsson.

remove pf_ioctl_head/pf_newif_head, which was never used.

PF from openbsd 3.5. missing features:
- pfsync (due to protocol # assignment issues)
- carp (not really a PF portion, but thought important to mention)
- PF and ALTQ are mutually-exclusive. this will be sorted out when
kjc@csl.sony.co.jp updates ALTQ and PF (and API inbetween)

reviewed by matt, christos, perry

torture-test is very welcomed.

branches: 1.1.1;
Initial revision

merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.4.70; 1.4.72; 1.4.74; 1.4.76; 1.4.78;
merge ktrace-lwp.

resolve conflicts. (pf from OpenBSD 3.6, kernel part)

branches: 1.2.2;
PF from openbsd 3.5. missing features:
- pfsync (due to protocol # assignment issues)
- carp (not really a PF portion, but thought important to mention)
- PF and ALTQ are mutually-exclusive. this will be sorted out when
kjc@csl.sony.co.jp updates ALTQ and PF (and API inbetween)

reviewed by matt, christos, perry

torture-test is very welcomed.

branches: 1.1.1;
Initial revision

sys: Drop redundant NULL check before m_freem(9)

m_freem(9) safely has accepted NULL argument at least since 4.2BSD:
https://www.tuhs.org/cgi-bin/utree.pl?file=4.2BSD/usr/src/sys/sys/uipc_mbuf.c

Compile-tested on amd64/ALL.

Suggested by knakahara@

branches: 1.22.24;
remove htons, it is pointless (thanks joerg@)

remove args from ip_randomid() (John D. Baker). When does this file get built?

branches: 1.20.6;
Adopt <net/if_stats.h>.

branches: 1.19.6;
Replace M_ALIGN and MH_ALIGN by m_align.

Use non-variadic function pointer in protosw::pr_input.

branches: 1.17.2;
Add missing BPF_D_OUT. Reported by John D. Baker.

Implement the BPF direction filter (BIOC[GS]DIRECTION). It provides backward
compatibility with BIOC[GS]SEESENT ioctl. The userland interface is the same
as FreeBSD.

This change also fixes a bug that the direction is misunderstand on some
environment by passing the direction to bpf_mtap*() instead of checking
m->m_pkthdr.rcvif.

branches: 1.15.16;
Replace ifp of ip_moptions and ip6_moptions with if_index

The motivation is the same as the mbuf's rcvif case; avoid having a pointer
of an ifnet object in ip_moptions and ip6_moptions, which is not MP-safe.

ip_moptions and ip6_moptions can be stored in a PCB for inet or inet6
that's life time is different from ifnet one and so an ifnet object can be
disappeared anytime we get it via them. Thus we need to look up an ifnet
object by if_index every time for safe.

Avoid storing a pointer of an interface in a mbuf

Having a pointer of an interface in a mbuf isn't safe if we remove big
kernel locks; an interface object (ifnet) can be destroyed anytime in any
packet processing and accessing such object via a pointer is racy. Instead
we have to get an object from the interface collection (ifindex2ifnet) via
an interface index (if_index) that is stored to a mbuf instead of an
pointer.

The change provides two APIs: m_{get,put}_rcvif_psref that use psref(9)
for sleep-able critical sections and m_{get,put}_rcvif that use
pserialize(9) for other critical sections. The change also adds another
API called m_get_rcvif_NOMPSAFE, that is NOT MP-safe and for transition
moratorium, i.e., it is intended to be used for places where are not
planned to be MP-ified soon.

The change adds some overhead due to psref to performance sensitive paths,
however the overhead is not serious, 2% down at worst.

Proposed on tech-kern and tech-net.

Introduce m_set_rcvif and m_reset_rcvif

The API is used to set (or reset) a received interface of a mbuf.
They are counterpart of m_get_rcvif, which will come in another
commit, hide internal of rcvif operation, and reduce the diff of
the upcoming change.

No functional change.

Constify rtentry of if_output

We no longer need to change rtentry below if_output.

The change makes it clear where rtentries are changed (or not)
and helps forthcoming locking (os psrefing) rtentries.

include "ioconf.h" to get the 'void <driver>attach(int count);' prototype.

branches: 1.10.6;
remove unused variable to avoid warning from gcc 4.8.

branches: 1.9.2; 1.9.4;
remove KAME IPSEC, replaced by FAST_IPSEC

do missing ipsec->kame_ipsec renames

branches: 1.7.8; 1.7.12;
ip_randomid: make mechanism MP-safe and more modular.

OK matt@

Push the bpf_ops usage back into bpf.h. Push the common ifp->if_bpf
check into the inline functions as well the fourth argument for
bpf_attach.

branches: 1.5.2; 1.5.4;
Fix a typo introduced by the bpf linkage change.

Redefine bpf linkage through an always present op vector, i.e.
#if NBPFILTER is no longer required in the client. This change
doesn't yet add support for loading bpf as a module, since drivers
can register before bpf is attached. However, callers of bpf can
now be modularized.

Dynamically loadable bpf could probably be done fairly easily with
coordination from the stub driver and the real driver by registering
attachments in the stub before the real driver is loaded and doing
a handoff. ... and I'm not going to ponder the depths of unload
here.

Tested with i386/MONOLITHIC, modified MONOLITHIC without bpf and rump.

Import pfsync support from OpenBSD 4.2

Pfsync interface exposes change in the pf(4) over a pseudo-interface, and can
be used to synchronise different pf.

This work was part of my 2009 GSoC

No objection on tech-net@

branches: 1.2.58;
PF from openbsd 3.5. missing features:
- pfsync (due to protocol # assignment issues)
- carp (not really a PF portion, but thought important to mention)
- PF and ALTQ are mutually-exclusive. this will be sorted out when
kjc@csl.sony.co.jp updates ALTQ and PF (and API inbetween)

reviewed by matt, christos, perry

torture-test is very welcomed.

branches: 1.1.1;
Initial revision

Use non-variadic function pointer in protosw::pr_input.

branches: 1.3.62; 1.3.64;
Import pfsync support from OpenBSD 4.2

Pfsync interface exposes change in the pf(4) over a pseudo-interface, and can
be used to synchronise different pf.

This work was part of my 2009 GSoC

No objection on tech-net@

branches: 1.2.58;
PF from openbsd 3.5. missing features:
- pfsync (due to protocol # assignment issues)
- carp (not really a PF portion, but thought important to mention)
- PF and ALTQ are mutually-exclusive. this will be sorted out when
kjc@csl.sony.co.jp updates ALTQ and PF (and API inbetween)

reviewed by matt, christos, perry

torture-test is very welcomed.

branches: 1.1.1;
Initial revision

inpcb: rename functions to in6pcb_*

inpcb: rename functions to inpcb_*

Inspired by rmind-smpnet patches.

Adjust pf, wg, dccp and sctp for struct inpcb integration

Clean up _LKM --> _MODULE leftovers.

Note that _KERNEL is always defined for modules.

Rename min/max -> uimin/uimax for better honesty.

These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.

HOWEVER! Some subsystems have

#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))

even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.

To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.

I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:

cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))

It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.

Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)

Rename

ip_undefer_csum -> in_undefer_cksum
in_delayed_cksum -> in_undefer_cksum_tcpudp

The two previous names were inconsistent and misleading.

Put the two functions into in_offload.c. Add comments to explain what
we're doing.

The same could be done for IPv6.

branches: 1.81.2;
Remove m_copy completely.

branches: 1.80.2;
It is normal for socket credentials to be missing for incoming sockets,
so don't warn.

PR/53036: Alexander Nasonov: 'block user' in pf's ruleset panics 8.0_BETA
Check for NULL.

Oh, what is this. Fix a remotely-triggerable integer overflow: the way we
define TCPOLEN_SACK makes it unsigned, and the comparison in the while()
is unsigned too. That's not the expected behavior, the original code
wanted a signed comparison.

It's pretty easy to make 'hlen' go negative and trigger a buffer overflow.

This bug was reported 8 years ago by Lucio Albornoz in PR/44059.

PR/52682: David Binderman: Fix wrong assignment (in the !__NetBSD__ code)

branches: 1.76.6;
Do ND in L2_output in the same manner as arpresolve

The benefits of this change are:
- The flow is consistent with IPv4 (and FreeBSD and OpenBSD)
- old: ip6_output => nd6_output (do ND if needed) => L2_output (lookup a stored cache)
- new: ip6_output => L2_output (lookup a cache. Do ND if cache not found)
- We can remove some workarounds in nd6_output
- We can move L2 specific operations to their own place
- The performance slightly improves because one cache lookup is reduced

branches: 1.75.2;
Add rtcache_unref to release points of rtentry stemming from rtcache

In the MP-safe world, a rtentry stemming from a rtcache can be freed at any
points. So we need to protect rtentries somehow say by reference couting or
passive references. Regardless of the method, we need to call some release
function of a rtentry after using it.

The change adds a new function rtcache_unref to release a rtentry. At this
point, this function does nothing because for now we don't add a reference
to a rtentry when we get one from a rtcache. We will add something useful
in a further commit.

This change is a part of changes for MP-safe routing table. It is separated
to avoid one big change that makes difficult to debug by bisecting.

branches: 1.74.2;
apply if_output_lock() to L3 callers which call ifp->if_output() of L2(or L3 tunneling).

Introduce m_set_rcvif and m_reset_rcvif

The API is used to set (or reset) a received interface of a mbuf.
They are counterpart of m_get_rcvif, which will come in another
commit, hide internal of rcvif operation, and reduce the diff of
the upcoming change.

No functional change.

branches: 1.72.2; 1.72.4; 1.72.6; 1.72.10;
Unbreak the build of pf

- Implement pktqueue interface for lockless IP input queue.
- Replace ipintrq and ip6intrq with the pktqueue mechanism.
- Eliminate kernel-lock from ipintr() and ip6intr().
- Some preparation work to push softnet_lock out of ipintr().

Discussed on tech-net.

branches: 1.70.2;
fix compiler warnings

branches: 1.69.2; 1.69.4;
remove KAME IPSEC, replaced by FAST_IPSEC

branches: 1.68.2; 1.68.6; 1.68.8;
do missing ipsec->kame_ipsec renames

branches: 1.67.2;
First step of random number subsystem rework described in
<20111022023242.BA26F14A158@mail.netbsd.org>. This change includes
the following:

An initial cleanup and minor reorganization of the entropy pool
code in sys/dev/rnd.c and sys/dev/rndpool.c. Several bugs are
fixed. Some effort is made to accumulate entropy more quickly at
boot time.

A generic interface, "rndsink", is added, for stream generators to
request that they be re-keyed with good quality entropy from the pool
as soon as it is available.

The arc4random()/arc4randbytes() implementation in libkern is
adjusted to use the rndsink interface for rekeying, which helps
address the problem of low-quality keys at boot time.

An implementation of the FIPS 140-2 statistical tests for random
number generator quality is provided (libkern/rngtest.c). This
is based on Greg Rose's implementation from Qualcomm.

A new random stream generator, nist_ctr_drbg, is provided. It is
based on an implementation of the NIST SP800-90 CTR_DRBG by
Henric Jungheim. This generator users AES in a modified counter
mode to generate a backtracking-resistant random stream.

An abstraction layer, "cprng", is provided for in-kernel consumers
of randomness. The arc4random/arc4randbytes API is deprecated for
in-kernel use. It is replaced by "cprng_strong". The current
cprng_fast implementation wraps the existing arc4random
implementation. The current cprng_strong implementation wraps the
new CTR_DRBG implementation. Both interfaces are rekeyed from
the entropy pool automatically at intervals justifiable from best
current cryptographic practice.

In some quick tests, cprng_fast() is about the same speed as
the old arc4randbytes(), and cprng_strong() is about 20% faster
than rnd_extract_data(). Performance is expected to improve.

The AES code in src/crypto/rijndael is no longer an optional
kernel component, as it is required by cprng_strong, which is
not an optional kernel component.

The entropy pool output is subjected to the rngtest tests at
startup time; if it fails, the system will reboot. There is
approximately a 3/10000 chance of a false positive from these
tests. Entropy pool _input_ from hardware random numbers is
subjected to the rngtest tests at attach time, as well as the
FIPS continuous-output test, to detect bad or stuck hardware
RNGs; if any are detected, they are detached, but the system
continues to run.

A problem with rndctl(8) is fixed -- datastructures with
pointers in arrays are no longer passed to userspace (this
was not a security problem, but rather a major issue for
compat32). A new kernel will require a new rndctl.

The sysctl kern.arandom() and kern.urandom() nodes are hooked
up to the new generators, but the /dev/*random pseudodevices
are not, yet.

Manual pages for the new kernel interfaces are forthcoming.

branches: 1.66.2;
build pf module with WARNS=3, and remove the need for -Wno-shadow

Reduces the resources demanded by TCP sessions in TIME_WAIT-state using
methods called Vestigial Time-Wait (VTW) and Maximum Segment Lifetime
Truncation (MSLT).

MSLT and VTW were contributed by Coyote Point Systems, Inc.

Even after a TCP session enters the TIME_WAIT state, its corresponding
socket and protocol control blocks (PCBs) stick around until the TCP
Maximum Segment Lifetime (MSL) expires. On a host whose workload
necessarily creates and closes down many TCP sockets, the sockets & PCBs
for TCP sessions in TIME_WAIT state amount to many megabytes of dead
weight in RAM.

Maximum Segment Lifetimes Truncation (MSLT) assigns each TCP session to
a class based on the nearness of the peer. Corresponding to each class
is an MSL, and a session uses the MSL of its class. The classes are
loopback (local host equals remote host), local (local host and remote
host are on the same link/subnet), and remote (local host and remote
host communicate via one or more gateways). Classes corresponding to
nearer peers have lower MSLs by default: 2 seconds for loopback, 10
seconds for local, 60 seconds for remote. Loopback and local sessions
expire more quickly when MSLT is used.

Vestigial Time-Wait (VTW) replaces a TIME_WAIT session's PCB/socket
dead weight with a compact representation of the session, called a
"vestigial PCB". VTW data structures are designed to be very fast and
memory-efficient: for fast insertion and lookup of vestigial PCBs,
the PCBs are stored in a hash table that is designed to minimize the
number of cacheline visits per lookup/insertion. The memory both
for vestigial PCBs and for elements of the PCB hashtable come from
fixed-size pools, and linked data structures exploit this to conserve
memory by representing references with a narrow index/offset from the
start of a pool instead of a pointer. When space for new vestigial PCBs
runs out, VTW makes room by discarding old vestigial PCBs, oldest first.
VTW cooperates with MSLT.

It may help to think of VTW as a "FIN cache" by analogy to the SYN
cache.

A 2.8-GHz Pentium 4 running a test workload that creates TIME_WAIT
sessions as fast as it can is approximately 17% idle when VTW is active
versus 0% idle when VTW is inactive. It has 103 megabytes more free RAM
when VTW is active (approximately 64k vestigial PCBs are created) than
when it is inactive.

branches: 1.64.2;
Add support for pfs(8)

pfs(8) is a tool similar to ipfs(8) but for pf(4). It allows the admin to
dump internal configuration of pf, and restore at a latter point, after a
maintenance reboot for example, in a transparent way for user.

This work has been done mostly during my GSoC 2009

No objections on tech-net@

- Make the pf and pflog driver able to detach.
- Add code for module support.

Original patch from Jared McNeill

Spello in comment.

branches: 1.61.2; 1.61.4;
Redefine bpf linkage through an always present op vector, i.e.
#if NBPFILTER is no longer required in the client. This change
doesn't yet add support for loading bpf as a module, since drivers
can register before bpf is attached. However, callers of bpf can
now be modularized.

Dynamically loadable bpf could probably be done fairly easily with
coordination from the stub driver and the real driver by registering
attachments in the stub before the real driver is loaded and doing
a handoff. ... and I'm not going to ponder the depths of unload
here.

Tested with i386/MONOLITHIC, modified MONOLITHIC without bpf and rump.

Replace uidinfo.h with kauth.h, should fix problems observed by tron@.

Use the right member to store gid in the non-NetBSD case.

Pointed out by uebayasi@ and cegger@, thanks!

Get uid/gid from the socket's credentials.

Import pfsync support from OpenBSD 4.2

Pfsync interface exposes change in the pf(4) over a pseudo-interface, and can
be used to synchronise different pf.

This work was part of my 2009 GSoC

No objection on tech-net@

Remove LKM code from pf.

Reduce diff with OpenBSD. No functional change.

Fix http://www.securityfocus.com/archive/1/502634, from OpenBSD.
XXX: should be pulled up to 5.x

branches: 1.53.2; 1.53.4; 1.53.8;
Move uidinfo to its own module in kern_uidinfo.c and include in rump.
No functional change to uidinfo.

branches: 1.52.2;
merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.51.2; 1.51.4; 1.51.6; 1.51.8;
Make ip6 and icmp6 stats per-cpu.

Make IP, TCP, UDP, and ICMP statistics per-CPU. The stats are collated
when the user requests them via sysctl.

Change ICMP6 stats from a structure to an array of uint64_t's.

Note: This is ABI-compatible with the old icmp6stat structure; old netstat
binaries will continue to work properly.

Change TCP stats from a structure to an array of uint64_t's.

Note: This is ABI-compatible with the old tcpstat structure; old netstat
binaries will continue to work properly.

Change IP stats from a structure to an array of uint64_t's.

Note: This is ABI-compatible with the old ipstat structure; old netstat
binaries will continue to work properly.

Change UDP stats from a structure to an array of uint64_t's.

Note: This is ABI-compatible with the old icmpstat structure; old netstat
binaries will continue to work properly.

Change ICMP stats from a structure to an array of uint64_t's.

Note: This is ABI-compatible with the old icmpstat structure; old netstat
binaries will continue to work properly.

branches: 1.44.6;
Change rtcache_init()+rtcache_getrt() and
rtcache_init_noclone()+rtcache_getrt() to single rtcache_init()
and rtcache_init_clone() calls.

Poison struct route->ro_rt uses in the kernel by changing the name
to _ro_rt. Use rtcache_getrt() to access a route cache's struct
rtentry *.

Introduce struct ifnet->if_dl that always points at the interface
identifier/link-layer address. Make code that treated the first
ifaddr on struct ifnet->if_addrlist as the interface address use
if_dl, instead.

Remove stale debugging code from net/route.c. Move the rtflush()
code into rtcache_clear() and delete rtflush(). Delete rtalloc(),
because nothing uses it any more.

Make ND6_HINT an inline, lowercase subroutine, nd6_hint.

I've done my best to convert IP Filter, the ISO stack, and the
AppleTalk stack to rtcache_getrt(). They compile, but I have not
tested them. I have given the changes to PF, GRE, IPv4 and IPv6
stacks a lot of exercise.

use __KERNEL_RCSID()

branches: 1.41.2; 1.41.4; 1.41.6;
Bug fix: make pf_route() set M_CSUM_IPV4 before calling ip_fragment().

If you use a route-to rule such as 'pass out quick on ath0 route-to
gre2 all', and the MTU on gre2 is smaller than the MTU on ath0,
then pf_route() will fragment your packet by calling ip_fragment().
Because pf_route() did not set M_CSUM_IPv4, ip_fragment() would
not compute the checksum on the fragments, and PF would send IP
fragments with bad checksums out of gre2.

branches: 1.40.2; 1.40.8;
reduce diff.

branches: 1.39.2; 1.39.6;
Coverity CID 3157: remove bogus break.

pfctl: extend pf.conf(5) syntax. Let the operator supply an optional
"state lock" flag (if-bound, gr-bound, floating) at the end of a
NAT rule. The new syntax is backwards-compatbile with the old
syntax.

PF (kernel): change the macro BOUND_IFACE() to the inline function
bound_iface(), and add a new argument, the applicable NAT rule.
Use both the flags on the applicable filter rule and on the applicable
NAT rule to decide whether or not to bind a state to the interface
or the group where it is created.

Eliminate address family-specific route caches (struct route, struct
route_in6, struct route_iso), replacing all caches with a struct
route.

The principle benefit of this change is that all of the protocol
families can benefit from route cache-invalidation, which is
necessary for correct routing. Route-cache invalidation fixes an
ancient PR, kern/3508, at long last; it fixes various other PRs,
also.

Discussions with and ideas from Joerg Sonnenberger influenced this
work tremendously. Of course, all design oversights and bugs are
mine.

DETAILS

1 I added to each address family a pool of sockaddrs. I have
introduced routines for allocating, copying, and duplicating,
and freeing sockaddrs:

struct sockaddr *sockaddr_alloc(sa_family_t af, int flags);
struct sockaddr *sockaddr_copy(struct sockaddr *dst,
const struct sockaddr *src);
struct sockaddr *sockaddr_dup(const struct sockaddr *src, int flags);
void sockaddr_free(struct sockaddr *sa);

sockaddr_alloc() returns either a sockaddr from the pool belonging
to the specified family, or NULL if the pool is exhausted. The
returned sockaddr has the right size for that family; sa_family
and sa_len fields are initialized to the family and sockaddr
length---e.g., sa_family = AF_INET and sa_len = sizeof(struct
sockaddr_in). sockaddr_free() puts the given sockaddr back into
its family's pool.

sockaddr_dup() and sockaddr_copy() work analogously to strdup()
and strcpy(), respectively. sockaddr_copy() KASSERTs that the
family of the destination and source sockaddrs are alike.

The 'flags' argumet for sockaddr_alloc() and sockaddr_dup() is
passed directly to pool_get(9).

2 I added routines for initializing sockaddrs in each address
family, sockaddr_in_init(), sockaddr_in6_init(), sockaddr_iso_init(),
etc. They are fairly self-explanatory.

3 structs route_in6 and route_iso are no more. All protocol families
use struct route. I have changed the route cache, 'struct route',
so that it does not contain storage space for a sockaddr. Instead,
struct route points to a sockaddr coming from the pool the sockaddr
belongs to. I added a new method to struct route, rtcache_setdst(),
for setting the cache destination:

int rtcache_setdst(struct route *, const struct sockaddr *);

rtcache_setdst() returns 0 on success, or ENOMEM if no memory is
available to create the sockaddr storage.

It is now possible for rtcache_getdst() to return NULL if, say,
rtcache_setdst() failed. I check the return value for NULL
everywhere in the kernel.

4 Each routing domain (struct domain) has a list of live route
caches, dom_rtcache. rtflushall(sa_family_t af) looks up the
domain indicated by 'af', walks the domain's list of route caches
and invalidates each one.

branches: 1.36.2; 1.36.4;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

In pf_rtlabel_match, use rtcache_free()/rtcache_init(). This is
just cosmetic, since the whole routine is presently #if 0'd.

branches: 1.34.2;
Introduce new helper functions to abstract the route caching.
rtcache_init and rtcache_init_noclone lookup ro_dst and store
the result in ro_rt, taking care of the reference counting and
calling the domain specific route cache.
rtcache_free checks if a route was cashed and frees the reference.
rtcache_copy copies ro_dst of the given struct route, checking that
enough space is available and incrementing the reference count of the
cached rtentry if necessary.
rtcache_check validates that the cached route is still up. If it isn't,
it tries to look it up again. Afterwards ro_rt is either a valid again
or NULL.
rtcache_copy is used internally.

Adjust to callers of rtalloc/rtflush in the tree to check the sanity of
ro_dst first (if necessary). If it doesn't fit the expectations, free
the cache, otherwise check if the cached route is still valid. After
that combination, a single check for ro_rt == NULL is enough to decide
whether a new lookup needs to be done with a different ro_dst.
Make the route checking in gre stricter by repeating the loop check
after revalidation.
Remove some unused RADIX_MPATH code in in6_src.c. The logic is slightly
changed here to first validate the route and check RTF_GATEWAY
afterwards. This is sementically equivalent though.
etherip doesn't need sc_route_expire similiar to the gif changes from
dyoung@ earlier.

Based on the earlier patch from dyoung@, reviewed and discussed with
him.

Don't apply a window scale to the window size in a SYN packet.

Here are various changes designed to protect against bad IPv4
routing caused by stale route caches (struct route). Route caches
are sprinkled throughout PCBs, the IP fast-forwarding table, and
IP tunnel interfaces (gre, gif, stf).

Stale IPv6 and ISO route caches will be treated by separate patches.

Thank you to Christoph Badura for suggesting the general approach
to invalidating route caches that I take here.

Here are the details:

Add hooks to struct domain for tracking and for invalidating each
domain's route caches: dom_rtcache, dom_rtflush, and dom_rtflushall.

Introduce helper subroutines, rtflush(ro) for invalidating a route
cache, rtflushall(family) for invalidating all route caches in a
routing domain, and rtcache(ro) for notifying the domain of a new
cached route.

Chain together all IPv4 route caches where ro_rt != NULL. Provide
in_rtcache() for adding a route to the chain. Provide in_rtflush()
and in_rtflushall() for invalidating IPv4 route caches. In
in_rtflush(), set ro_rt to NULL, and remove the route from the
chain. In in_rtflushall(), walk the chain and remove every route
cache.

In rtrequest1(), call rtflushall() to invalidate route caches when
a route is added.

In gif(4), discard the workaround for stale caches that involves
expiring them every so often.

Replace the pattern 'RTFREE(ro->ro_rt); ro->ro_rt = NULL;' with a
call to rtflush(ro).

Update ipflow_fastforward() and all other users of route caches so
that they expect a cached route, ro->ro_rt, to turn to NULL.

Take care when moving a 'struct route' to rtflush() the source and
to rtcache() the destination.

In domain initializers, use .dom_xxx tags.

KNF here and there.

Indent these macros for readability. People have to read this
code, too.

Lightly constify. Helps compile-time checking that we are not
scribbling over shared or read-only memory---e.g., in mbufs.

No need for a struct route_in6 in pf_route6(). Replace it with a
sockaddr_in6.

In pf_calc_mss(), factor common code out of PF_INET and PF_INET6
switch cases.

branches: 1.28.2; 1.28.8;
__unused removal on arguments; approved by core.

Merge the peter-altq branch.

(sync with KAME & add support for using ALTQ with pf(4)).

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

PR/34746: Nino Dehne: pf(4)'s synproxy state breaks when used with tags

Apply OpenBSD src/sys/net/pf.c rev 1.486 and 1.487:

1.486:
When synproxy sends packets to the destination host, make sure to copy
the 'tag' from the original state entry into the outgoing mbuf.

1.487:
When synproxy completes the replayed handshake and modifies the state
into a normal one, it sets both peers' sequence windows. Fix a bug where
the previously advertised windows are applied to the wrong side (i.e.
peer A's seqhi is peer A's seqlo plus peer B's, not A's, window). This
went undetected because mostly the windows are similar and/or re-
advertised soon. But there are (rare) cases where a synproxy'd connection
would stall right after handshake. Found by Gleb Smirnoff.

In pf, there are lots of #ifdef ALTQ, but our ALTQ is not what pf expects,
and if ALTQ and pf are both enabled, it leads to compile errors. So,
change all tests for ALTQ to ALTQ_NEW, which won't be defined.

This allows simultaneous compilation of pf and ALTQ and is a temporary
measure before the peter-altq brach is merged.

Tested and approved by Peter Postma.

branches: 1.23.8; 1.23.10;
XXX: GCC uninitialized

quell GCC 4.1 uninitialised variable warnings.

XXX: we should audit the tree for which old ones are no longer needed
after getting the older compilers out of the tree..

branches: 1.21.2; 1.21.4; 1.21.6;
Fix TCP/UDP checksum handling as pointed out by Daniel Hartmeier in:
http://mail-index.netbsd.org/tech-net/2006/01/21/0000.html.

Problem reported and patch tested by der Mouse & Nino Dehne (PR/32874).

In pf_socket_lookup() fix copy & paste problem when in6_pcblookup_bind()
returns NULL.

branches: 1.19.2; 1.19.4; 1.19.6;
merge ktrace-lwp.

Adjust for icmp_error signature.

branches: 1.17.2; 1.17.4;
Resolve conflicts (pf from OpenBSD 3.7, kernel part).

Use an "XXXGCC -Wuninitalized" style that is consistent with that used
elsewhere in the tree.

Cleanup XXGCC in a few places to make it easier to see.

Fix unitialized warnings that only crop up on m68k. XXGCC taggedd

more fallout from so_uid -> so_uidinfo.

branches: 1.12.4;
Merge in a fix from OPENBSD_3_6.
ok yamt@

> MFC:
> Fix by dhartmei@
>
> ICMP state entries use the ICMP ID as port for the unique state key. When
> checking for a usable key, construct the key in the same way. Otherwise,
> a colliding key might be missed or a state insertion might be refused even
> though it could be inserted. The second case triggers the endless loop
> fixed by 1.474, possibly allowing a NATed LAN client to lock up the kernel.
> Report and test data by Srebrenko Sehic.

branches: 1.11.2; 1.11.4;
Apply a patch from OPENBSD_3_6 branch (ok yamt).

MFC:
Fix by dhartmei@

IPv6 packets can contain headers (like options) before the TCP/UDP/ICMP6
header. pf finds the first TCP/UDP/ICMP6 header to filter by traversing
the header chain. In the case where headers are skipped, the protocol
checksum verification used the wrong length (included the skipped headers),
leading to incorrectly mismatching checksums. Such IPv6 packets with
headers were silently dropped. Reported by Bernhard Schmidt.

ok deraadt@ dhartmei@ mcbride@

Apply a patch from OPENBSD_3_6 branch (ok yamt).

MFC:
Fix by mcbride@

Initialise init_addr in pf_map_addr() in the PF_POOL_ROUNDROBIN,
prevents a possible endless loop in pf_get_sport() with 'static-port'

Reported by adm at celeritystorm dot com in FreeBSD PR74930, debugging
by dhartmei@

ok mcbride@ dhartmei@ deraadt@ henning@

pf_check_proto_cksum: use {tcp,udp}_input_checksum so that we can:
- handle loopback checksum omission properly.
- profit from h/w checksum offloading.

Apply a patch from OpenBSD 3.6 branch (ok yamt@).

MFC:
Fix by dhartmei@

fix a bug that leads to a crash when binat rules of the form
'binat from ... to ... -> (if)' are used, where the interface
is dynamic. reported by kos(at)bastard(dot)net, analyzed by
Pyun YongHyeon.

Apply a patch from the OPENBSD_3_6 branch, ok itojun.

MFC:
Fix by dhartmei@

The flag to re-filter pf-generated packets was set wrong by synproxy
for ACKs. It should filter the ACK replayed to the server, instead of
of the one to the client.

Apply a patch from the OPENBSD_3_6 branch, ok itojun.

MFC:
Fix by dhartmei@

For RST generated due to state mismatch during handshake, don't set
th_flags TH_ACK and leave th_ack 0, just like the RST generated by
the stack in this case. Fixes the Raptor workaround.

resolve conflicts. (pf from OpenBSD 3.6, kernel part)

remove no longer needed caddr_t casts to reduce diffs from openbsd.

branches: 1.3.2;
Fix formatting for 64 bit archs. This fixes PR port-sparc64/26010.
While there, make it compile for non-INET6 aware kernels.

PF from openbsd 3.5. missing features:
- pfsync (due to protocol # assignment issues)
- carp (not really a PF portion, but thought important to mention)
- PF and ALTQ are mutually-exclusive. this will be sorted out when
kjc@csl.sony.co.jp updates ALTQ and PF (and API inbetween)

reviewed by matt, christos, perry

torture-test is very welcomed.

branches: 1.1.1;
Initial revision

Use if_acquire and if_release instead of using psref API directly

- Provide if_release for consistency to if_acquire
- Use if_acquire and if_release for ifp iterations
- Make ifnet_psref_class static

branches: 1.32.2;
pfil(9) improvements to handle address changes:

Add:
PFIL_IFADDR call on interface reconfig (mbuf is ioctl #)
PFIL_IFNET call on interface attach/detach (mbuf is PFIL_IFNET_*)

from rmind@

Apply pserialize to some iterations of IP address lists

branches: 1.30.2;
Switch the address list of intefaces to pslist(9)

As usual, we leave the old list to avoid breaking kvm(3) users.

Remove unnecessary NULL checks of ifa->ifa_addr

If it's NULL, it should be a bug. There many IFADDR_FOREACH that don't do
NULL check. If it can be NULL, they should fire already.

Use curlwp_bind and curlwp_bindx instead of open-coding LP_BOUND

Protect ifnet list with psz and psref

The change ensures that ifnet objects in the ifnet list aren't freed during
list iterations by using pserialize(9) and psref(9).

Note that the change adds a pslist(9) for ifnet but doesn't remove the
original ifnet list (ifnet_list) to avoid breaking kvm(3) users. We
shouldn't use the original list in the kernel anymore.

branches: 1.26.4;
Fix previous.

- Move IFNET_*() macros under #ifdef _KERNEL.
- Replace TAILQ_FOREACH on ifnet with IFNET_FOREACH().

branches: 1.24.4;
PFIL_HOOKS is dead.

Fix pf module build. Adjust pfil_remove_hook 3rd arguments.

Update pf to pfil(9) changes. Missed in previous commit.

branches: 1.21.8; 1.21.18; 1.21.22;
- Make the pf and pflog driver able to detach.
- Add code for module support.

Original patch from Jared McNeill

branches: 1.20.2; 1.20.4;
If pfi_address_add() has to extend the buffer, copy the data in the
right direction!
Fixes PR/41939.

Remove LKM code from pf.

use M_ZERO on malloc() and remove subsequent bzero().

pass M_NOWAIT instead of M_DONTWAIT to malloc.

branches: 1.16.4;
merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.15.6; 1.15.8; 1.15.10; 1.15.12; 1.15.14;
Use TAILQ_FOREACH().

use __KERNEL_RCSID()

branches: 1.13.2; 1.13.4;
Use IFADDR_FOREACH().

branches: 1.12.12; 1.12.14; 1.12.20; 1.12.22;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.

branches: 1.11.2;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.10.26;
merge ktrace-lwp.

branches: 1.9.2;
Resolve conflicts (pf from OpenBSD 3.7, kernel part).

Improve the cleanup routines for detachment. Fixes PR 28132.

Reviewed by yamt.

plug pfik_ifaddrhooks leaks by embedding it to pfi_kif.

resolve conflicts. (pf from OpenBSD 3.6, kernel part)

branches: 1.5.2;
fix dynaddr tracking.

from Peter Postma, PR/26369.
ok'ed by itojun.

ANSIfy. (inside #ifdef __NetBSD__)

from Peter Postma.
ok'ed by itojun.

make PF lkm working. from Peter Postma and Joel Wilsson.

remove pf_ioctl_head/pf_newif_head, which was never used.

PF from openbsd 3.5. missing features:
- pfsync (due to protocol # assignment issues)
- carp (not really a PF portion, but thought important to mention)
- PF and ALTQ are mutually-exclusive. this will be sorted out when
kjc@csl.sony.co.jp updates ALTQ and PF (and API inbetween)

reviewed by matt, christos, perry

torture-test is very welcomed.

branches: 1.1.1;
Initial revision

driver(9): devsw_detach never fails. Make it return void.

Prune a whole lotta dead branches as a result of this. (Some logic
calling this is also wrong for other reasons; devsw_detach is final
-- you should never have any reason to decide to roll it back. To be
cleaned up in subsequent commits...)

XXX kernel ABI change to devsw_detach signature requires bump

Explicitly cast pointers to uintptr_t before casting to enums. They are
not necessarily the same size. Don't cast pointers to bool, check for
NULL instead.

branches: 1.56.6;
Fix compilation of PF/IPF...

Rename

ip6_undefer_csum -> in6_undefer_cksum
in6_delayed_cksum -> in6_undefer_cksum_tcpudp

The two previous names were inconsistent and misleading.

Put the two functions into in6_offload.c. Add comments to explain what
we're doing.

Same as IPv4.

Fix build. pf_ioctl.c needs netinet/in_offload.h (after previous change).
Because this is in a module, apparently, that means that netinet_in_offload.h
needs to get installed in /usr/include, so do that as well.

Feel free to fix this in a better way...

Rename

ip_undefer_csum -> in_undefer_cksum
in_delayed_cksum -> in_undefer_cksum_tcpudp

The two previous names were inconsistent and misleading.

Put the two functions into in_offload.c. Add comments to explain what
we're doing.

The same could be done for IPv6.

branches: 1.52.2; 1.52.4;
Defer initialization of pf_status.host_id

The call to cprng_fast32() requires that per-cpu data has been initialized
by corng_fast_init(), which doesn't get called until after the first part
of auto-configuration is done, long after pfattach() calls cprng_fast32().

Fixed PR kern/52620

XXX This needs pull-up to the -8 branch.

branches: 1.51.8; 1.51.10;
include "ioconf.h" to get the 'void <driver>attach(int count);' prototype.

branches: 1.50.4;
Add d_discard to all struct cdevsw instances I could find.

All have been set to "nodiscard"; some should get a real implementation.

branches: 1.49.2;
Change (mostly mechanically) every cdevsw/bdevsw I can find to use
designated initializers.

I have not built every extant kernel so I have probably broken at
least one build; however I've also found and fixed some wrong
cdevsw/bdevsw entries so even if so I think we come out ahead.

PFIL_HOOKS is dead.

Update pf to pfil(9) changes. Missed in previous commit.

branches: 1.46.8; 1.46.12;
Remove arc4random() and arc4randbytes() from the kernel API. Replace
arc4random() hacks in rump with stubs that call the host arc4random() to
get numbers that are hopefully actually random (arc4random() keyed with
stack junk is not). This should fix some of the currently failing anita
tests -- we should no longer generate duplicate "random" MAC addresses in
the test environment.

branches: 1.45.2;
fix -Wshadow warnings when ALTQ is enabled

build pf module with WARNS=3, and remove the need for -Wno-shadow

make sure the "overload_tbl" member of "struct pf_rule" copied in
from userland is initialized (it is used by the kernel only)
fixes crash or data injection (CVE-2010-3830), usually by root user only
OpenBSD has rewritten the code to start with a zero'd struct and fills
in needed parts only - to be considered in case a newer pf version
is imported.

branches: 1.42.2;
Add support for pfs(8)

pfs(8) is a tool similar to ipfs(8) but for pf(4). It allows the admin to
dump internal configuration of pf, and restore at a latter point, after a
maintenance reboot for example, in a transparent way for user.

This work has been done mostly during my GSoC 2009

No objections on tech-net@

Do not unload pf when enabled, not even manually.

change module class to driver.

Do not auto unload pf if it's enabled.

- Make the pf and pflog driver able to detach.
- Add code for module support.

Original patch from Jared McNeill

branches: 1.37.2; 1.37.4;
Move firewall/NAT policy back to respective subsystems (pf, ipf).

Note: the ipf code contains a lot of ifdefs, some of them for NetBSD
versions that are no longer maintained. It won't make the code more
readable, but we should consider removing them.

Import pfsync support from OpenBSD 4.2

Pfsync interface exposes change in the pf(4) over a pseudo-interface, and can
be used to synchronise different pf.

This work was part of my 2009 GSoC

No objection on tech-net@

Remove LKM code from pf.

Wrap definition of pfil6_wrapper in #ifdef INET6.

From Scott Ellis in PR/39007.

merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.32.8; 1.32.10; 1.32.12; 1.32.14; 1.32.16;
use __KERNEL_RCSID()

branches: 1.31.8; 1.31.16; 1.31.18; 1.31.20;
Merge some of the less invasive changes from the vmlocking branch:

- kthread, callout, devsw API changes
- select()/poll() improvements
- miscellaneous MT safety improvements

branches: 1.30.2;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.

branches: 1.29.2;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.28.4;
__unused removal on arguments; approved by core.

Merge the peter-altq branch.

(sync with KAME & add support for using ALTQ with pf(4)).

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

In pf, there are lots of #ifdef ALTQ, but our ALTQ is not what pf expects,
and if ALTQ and pf are both enabled, it leads to compile errors. So,
change all tests for ALTQ to ALTQ_NEW, which won't be defined.

This allows simultaneous compilation of pf and ALTQ and is a temporary
measure before the peter-altq brach is merged.

Tested and approved by Peter Postma.

Remove ugly (void *) casts from network scope authorization wrapper and
calls to it.

While here, adapt code for system scope listeners to avoid some more
casts (forgotten in previous run).

Update documentation.

branches: 1.23.2;
First take at security model abstraction.

- Add a few scopes to the kernel: system, network, and machdep.

- Add a few more actions/sub-actions (requests), and start using them as
opposed to the KAUTH_GENERIC_ISSUSER place-holders.

- Introduce a basic set of listeners that implement our "traditional"
security model, called "bsd44". This is the default (and only) model we
have at the moment.

- Update all relevant documentation.

- Add some code and docs to help folks who want to actually use this stuff:

* There's a sample overlay model, sitting on-top of "bsd44", for
fast experimenting with tweaking just a subset of an existing model.

This is pretty cool because it's *really* straightforward to do stuff
you had to use ugly hacks for until now...

* And of course, documentation describing how to do the above for quick
reference, including code samples.

All of these changes were tested for regressions using a Python-based
testsuite that will be (I hope) available soon via pkgsrc. Information
about the tests, and how to write new ones, can be found on:

http://kauth.linbsd.org/kauthwiki

NOTE FOR DEVELOPERS: *PLEASE* don't add any code that does any of the
following:

- Uses a KAUTH_GENERIC_ISSUSER kauth(9) request,
- Checks 'securelevel' directly,
- Checks a uid/gid directly.

(or if you feel you have to, contact me first)

This is still work in progress; It's far from being done, but now it'll
be a lot easier.

Relevant mailing list threads:

http://mail-index.netbsd.org/tech-security/2006/01/25/0011.html
http://mail-index.netbsd.org/tech-security/2006/03/24/0001.html
http://mail-index.netbsd.org/tech-security/2006/04/18/0000.html
http://mail-index.netbsd.org/tech-security/2006/05/15/0000.html
http://mail-index.netbsd.org/tech-security/2006/08/01/0000.html
http://mail-index.netbsd.org/tech-security/2006/08/25/0000.html

Many thanks to YAMAMOTO Takashi, Matt Thomas, and Christos Zoulas for help
stablizing kauth(9).

Full credit for the regression tests, making sure these changes didn't break
anything, goes to Matt Fleming and Jaime Fournier.

Happy birthday Randi! :)

branches: 1.22.2;
add missing initializer

branches: 1.21.4; 1.21.8; 1.21.12;
merge ktrace-lwp.

pfil6_wrapper: handle M_CSUM_TCPv6|M_CSUM_UDPv6.

wrap INET only code by #if defined(INET). (in __NetBSD__ part)

pf_test() can set *mp to NULL, check for this before de-referencing it.
From Akihiro Sagawa in PR/30835.

branches: 1.17.2;
Resolve conflicts (pf from OpenBSD 3.7, kernel part).

branches: 1.16.2;
Fix a GCC warning when compiling on evbppc.
From FUKAUMI Naoki in PR #29669.

Merge in a fix from OPENBSD_3_6.
ok yamt@

> MFC:
> Fix by dhartmei@
>
> replace finer-grained spl locking in pfioctl() with a single broad lock
> around the entire body. this resolves the (misleading) panics in
> pf_tag_packet() during heavy ioctl operations (like when using authpf)
> that occur because softclock can interrupt ioctl on i386 since SMP.
> patch from camield@.

branches: 1.14.2; 1.14.4;
pfil4_wrapper: clear M_CANFASTFWD which is not compatible with pf.

Improve the cleanup routines for detachment. Fixes PR 28132.

Reviewed by yamt.

resolve conflicts. (pf from OpenBSD 3.6, kernel part)

backout whitespace changes to make further import easier.

pfil4_wrapper, pfil6_wrapper:
ensure that mbufs are writable beforehand as pf assumes it.
PR/26433.

branches: 1.9.2;
- rename PFIL_NEWIF to PFIL_IFNET, and handle interface detach events
as well.
- use it for pf(4).

mostly from Peter Postma. PR/26403.

fix dynaddr tracking.

from Peter Postma, PR/26369.
ok'ed by itojun.

call PFIL_NEWIF hooks at a correct place.
(on SIOCAIFADDR rather than SIOCGIFALIAS.)

from Peter Postma, PR/26402.
ok'ed by itojun.

make PF lkm working. from Peter Postma and Joel Wilsson.

remove pf_ioctl_head/pf_newif_head, which was never used.

PR kern/26011: pf leaks mbufs on disallowed packets. Peter Postma

Make it compile on non-IPv6 kernels.

add a pfdetach() method to be used by lkm's

PF from openbsd 3.5. missing features:
- pfsync (due to protocol # assignment issues)
- carp (not really a PF portion, but thought important to mention)
- PF and ALTQ are mutually-exclusive. this will be sorted out when
kjc@csl.sony.co.jp updates ALTQ and PF (and API inbetween)

reviewed by matt, christos, perry

torture-test is very welcomed.

branches: 1.1.1;
Initial revision

Remove the 't' argument from m_tag_find().

branches: 1.2.2; 1.2.4; 1.2.86; 1.2.88;
merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.1.1; 1.1.2; 1.1.4; 1.1.6;
file pf_mtag.c was initially added on branch yamt-pf42.

branches: 1.2.2; 1.2.4;
merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.1.1; 1.1.2; 1.1.4; 1.1.6;
file pf_mtag.h was initially added on branch yamt-pf42.

fix indentation issues.

found by GCC 12.

Adjust for fewer args in calling functions

branches: 1.28.30;
<sys/rnd.h> not needed for pf_norm.c.

branches: 1.27.6;
fix compiler warnings

branches: 1.26.8; 1.26.12;
Remove arc4random() and arc4randbytes() from the kernel API. Replace
arc4random() hacks in rump with stubs that call the host arc4random() to
get numbers that are hopefully actually random (arc4random() keyed with
stack junk is not). This should fix some of the currently failing anita
tests -- we should no longer generate duplicate "random" MAC addresses in
the test environment.

branches: 1.25.2;
build pf module with WARNS=3, and remove the need for -Wno-shadow

fix an uninitialised variable problem. large-ish function, but i
couldn't see how GCC 4.5 isn't wrong about this one.

ip_randomid: make mechanism MP-safe and more modular.

OK matt@

- Make the pf and pflog driver able to detach.
- Add code for module support.

Original patch from Jared McNeill

branches: 1.21.2; 1.21.4;
Remove LKM code from pf.

make this compile

branches: 1.19.4;
merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.18.6; 1.18.8; 1.18.10; 1.18.12; 1.18.14;
Pass 0 to ip_randomid since we don't know the salt.

use __KERNEL_RCSID()

branches: 1.16.14; 1.16.20; 1.16.22; 1.16.24; 1.16.26;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.

branches: 1.15.4; 1.15.8;
__unused removal on arguments; approved by core.

- sprinkle __unused on function decls.
- fix a couple of unused bugs
- no more -Wno-unused for i386

branches: 1.13.8; 1.13.10;
caddr_t -> u_char *, to match the variable type

branches: 1.12.2;
Use the SI capitalization for "Hz", "kHz", and "MHz" in comments and strings.
Add a space between numbers and Hz unit.

branches: 1.11.2; 1.11.4; 1.11.6; 1.11.8;
apply a fix from OpenBSD:

> revision 1.104
> date: 2006/01/18 22:03:21; author: dhartmei; state: Exp; lines: +2 -2
> fix a bug in the fragment cache (used for 'scrub fragment crop/drop-ovl',
> but not 'fragment reassemble'), which can cause some fragments to get
> inserted into the cache twice, thereby violating an invariant, and panic-
> ing the system subsequently. ok deraadt@

branches: 1.10.2;
merge ktrace-lwp.

branches: 1.9.2;
Resolve conflicts (pf from OpenBSD 3.7, kernel part).

pf_reassemble: clear stale csum_flags.

branches: 1.7.10;
Improve the cleanup routines for detachment. Fixes PR 28132.

Reviewed by yamt.

resolve conflicts. (pf from OpenBSD 3.6, kernel part)

backout whitespace changes to make further import easier.

remove no longer needed caddr_t casts to reduce diffs from openbsd.

branches: 1.3.2;
make PF lkm working. from Peter Postma and Joel Wilsson.

remove pf_ioctl_head/pf_newif_head, which was never used.

PF from openbsd 3.5. missing features:
- pfsync (due to protocol # assignment issues)
- carp (not really a PF portion, but thought important to mention)
- PF and ALTQ are mutually-exclusive. this will be sorted out when
kjc@csl.sony.co.jp updates ALTQ and PF (and API inbetween)

reviewed by matt, christos, perry

torture-test is very welcomed.

branches: 1.1.1;
Initial revision

use in6_print

Make ip6_sprintf(), in_fmtaddr(), lla_snprintf() and icmp6_redirect_diag() mpsafe.

Reviewed by ozaki-r@

branches: 1.11.14; 1.11.32; 1.11.36; 1.11.40;
remove unused expression

branches: 1.10.2;
- Make the pf and pflog driver able to detach.
- Add code for module support.

Original patch from Jared McNeill

branches: 1.9.2; 1.9.4;
Remove LKM code from pf.

merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.7.8; 1.7.10; 1.7.12; 1.7.14; 1.7.16;
use __KERNEL_RCSID()

branches: 1.6.14; 1.6.22; 1.6.24; 1.6.26;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.

branches: 1.5.26; 1.5.30;
merge ktrace-lwp.

branches: 1.4.12;
resolve conflicts. (pf from OpenBSD 3.6, kernel part)

branches: 1.3.2;
make PF lkm working. from Peter Postma and Joel Wilsson.

remove pf_ioctl_head/pf_newif_head, which was never used.

PF from openbsd 3.5. missing features:
- pfsync (due to protocol # assignment issues)
- carp (not really a PF portion, but thought important to mention)
- PF and ALTQ are mutually-exclusive. this will be sorted out when
kjc@csl.sony.co.jp updates ALTQ and PF (and API inbetween)

reviewed by matt, christos, perry

torture-test is very welcomed.

branches: 1.1.1;
Initial revision

branches: 1.2.2; 1.2.4;
merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.1.1; 1.1.2; 1.1.4; 1.1.6;
file pf_ruleset.c was initially added on branch yamt-pf42.

In pfr_fix_anchor(), change an overlapping bcopy() call to a memmove()
call.

branches: 1.18.16;
Don't invoke UB.

Heads up by John D. Baker.

Commit the patch from
<http://mail-index.netbsd.org/current-users/2010/09/12/msg014289.html>,
fixing a "panic: pool 'pfrktable' is IPL_NONE, but called from
interrupt context" that occurred on NetBSD/sparc.

branches: 1.16.2;
- Make the pf and pflog driver able to detach.
- Add code for module support.

Original patch from Jared McNeill

branches: 1.15.2; 1.15.4;
Remove LKM code from pf.

merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.13.8; 1.13.10; 1.13.12; 1.13.14; 1.13.16;
use __KERNEL_RCSID()

branches: 1.12.14; 1.12.22; 1.12.24; 1.12.26;
Pass an ipl argument to pool_init/POOL_INIT to be used when initializing
the pool's lock.

branches: 1.11.2;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.10.2;
Make code concise by removing uninformative #ifdef's.

branches: 1.9.6; 1.9.8;
Initialize h4 and h6 to NULL.
Fixes a panic reported by Mipam on -current-users.

Fix strict aliasing issues and while I am here fix a memory leak on error

branches: 1.7.4; 1.7.6; 1.7.8; 1.7.12; 1.7.14;
merge ktrace-lwp.

branches: 1.6.2;
Resolve conflicts (pf from OpenBSD 3.7, kernel part).

resolve conflicts. (pf from OpenBSD 3.6, kernel part)

pull following fixes from openbsd. ok'ed by itojun.

> ----------------------------
> revision 1.58
> date: 2004/06/23 04:34:17; author: mcbride; state: Exp; lines: +5 -3
> pfr_commit_ktable calls functions that can result in the current
> ktable being destroyed, which makes it unsafe in a SLIST_FOREACH.
>
> Fix from Chris Pascoe
> ----------------------------
> revision 1.56
> date: 2004/06/11 05:21:20; author: mcbride; state: Exp; lines: +5 -3
> Eliminate a dereference after pool_put when an inactive/no-longer referenced
> table is destroyed in pfr_setflags_ktable.
>
> Fix from Chris Pascoe
> ----------------------------

branches: 1.3.2;
make PF lkm working. from Peter Postma and Joel Wilsson.

remove pf_ioctl_head/pf_newif_head, which was never used.

PF from openbsd 3.5. missing features:
- pfsync (due to protocol # assignment issues)
- carp (not really a PF portion, but thought important to mention)
- PF and ALTQ are mutually-exclusive. this will be sorted out when
kjc@csl.sony.co.jp updates ALTQ and PF (and API inbetween)

reviewed by matt, christos, perry

torture-test is very welcomed.

branches: 1.1.1;
Initial revision

Need opt_inet.h for #ifdef INET, INET6.

branches: 1.22.28; 1.22.32;
- Implement pktqueue interface for lockless IP input queue.
- Replace ipintrq and ip6intrq with the pktqueue mechanism.
- Eliminate kernel-lock from ipintr() and ip6intr().
- Some preparation work to push softnet_lock out of ipintr().

Discussed on tech-net.

branches: 1.21.2;
fix compiler warnings

branches: 1.20.6; 1.20.10;
protect "union sockaddr_union" from being defined twice by a CPP symbol
(copied from FreeBSD), allows coexistence of (FAST_)IPSEC and pf

branches: 1.19.8; 1.19.12;
Add support for pfs(8)

pfs(8) is a tool similar to ipfs(8) but for pf(4). It allows the admin to
dump internal configuration of pf, and restore at a latter point, after a
maintenance reboot for example, in a transparent way for user.

This work has been done mostly during my GSoC 2009

No objections on tech-net@

- Make the pf and pflog driver able to detach.
- Add code for module support.

Original patch from Jared McNeill

branches: 1.17.2; 1.17.4;
Remove LKM code from pf.

merge yamt-pf42 branch.
(import newer pf from OpenBSD 4.2)

ok'ed by peter@. requested by core@

branches: 1.15.22; 1.15.24; 1.15.26; 1.15.28; 1.15.30;
reduce diff.

branches: 1.14.2; 1.14.10; 1.14.14;
Kill caddr_t; there will be some MI fallout, but it will be fixed shortly.

branches: 1.13.2;
Lightly constify. Helps compile-time checking that we are not
scribbling over shared or read-only memory---e.g., in mbufs.

branches: 1.12.6; 1.12.8;
merge FreeBSD timecounters from branch simonb-timecounters
- struct timeval time is gone
time.tv_sec -> time_second
- struct timeval mono_time is gone
mono_time.tv_sec -> time_uptime
- access to time via
{get,}{micro,nano,bin}time()
get* versions are fast but less precise
- support NTP nanokernel implementation (NTP API 4)
- further reading:
Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf
NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html

branches: 1.11.2; 1.11.4; 1.11.6; 1.11.10; 1.11.12;
Include netinet/in.h, for compatibility with OpenBSD (we #ifdef'ed out a
header which includes netinet/in.h on OpenBSD).

Pointed out by Thomas E. Spanjaard.
No objection from yamt@.

branches: 1.10.2;
merge ktrace-lwp.

branches: 1.9.2;
Resolve conflicts (pf from OpenBSD 3.7, kernel part).

Don't put the hook definitions into #ifdef _KERNEL.

(needed to compile pf programs because of the previous change)

plug pfik_ifaddrhooks leaks by embedding it to pfi_kif.

resolve conflicts. (pf from OpenBSD 3.6, kernel part)

"RB_PROTOTYPE();" does not lint because you end up with two
consecutive semicolons, so let's use RB_PROTOTYPE() alone.

branches: 1.4.2;
- rename PFIL_NEWIF to PFIL_IFNET, and handle interface detach events
as well.
- use it for pf(4).

mostly from Peter Postma. PR/26403.

make PF lkm working. from Peter Postma and Joel Wilsson.

remove pf_ioctl_head/pf_newif_head, which was never used.

PF from openbsd 3.5. missing features:
- pfsync (due to protocol # assignment issues)
- carp (not really a PF portion, but thought important to mention)
- PF and ALTQ are mutually-exclusive. this will be sorted out when
kjc@csl.sony.co.jp updates ALTQ and PF (and API inbetween)

reviewed by matt, christos, perry

torture-test is very welcomed.

branches: 1.1.1;
Initial revision