kernel: extract rules, lookup socket, process filtering, reviews by christos@

branches: 1.30.6;
bsd.own.mk: rename to CC_WNO_ADDRESS_OF_PACKED_MEMBER

Provide a single variable
CC_WNO_ADDRESS_OF_PACKED_MEMBER
with options for both clang and gcc, to replace
CLANG_NO_ADDR_OF_PACKED_MEMBER
CC_NO_ADDR_OF_PACKED_MEMBER
GCC_NO_ADDR_OF_PACKED_MEMBER

Using the convention CC_compilerflag, where compilerflag
is based on the full compiler flag name.

bsd.own.mk: rename NO_ADDR_OF_PACKED_MEMBER to CC_NO_ADDR_OF_PACKED_MEMBER

Add "CC_" prefix to NO_ADDR_OF_PACKED_MEMBER.

We now have the following convention:
GCC_NO_*: Disable specific warnings from GCC.
CLANG_NO_*: Disable specific warnings from Clang.
CC_NO_*: Disable specific warnings from both Clang and GCC.

This reserves "NO_*" Makefile variables to disable other features.

Some space->tab fixes.

Provide a global no-packed-address for both gcc and clang

use the GCC_ warning variable

Avoid alignment of packed member

kill WARNS = 3

Update the NPF kernel module Makefile. Missed in the previous commit.

Bump default value of WARNS for modules from 3 to 5, and
explicitly set WARNS for modules that fail with WARNS=5.

Also, turn on -Wno-missing-noreturn for clang for some files.

At the moment, among ~ 360 modules,
- 2 (lua and zfs) need WARNS=0
- 1 (solaris) needs WARNS=1
- 136 need WARNS=3 (mostly due to sign-compare)
- 4 need WARNS=4
- others can be compiled with WARNS=5

Discussed on tech-kern.

NPF: Major rework -- migrate NPF to the libnv library.
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.

Thanks to christos@ for extra testing.

branches: 1.21.14; 1.21.16;
NPF: implement dynamic handling of interface addresses (the kernel part).

oops forgot to commit this one.

ditch ptree and use lpm

branches: 1.18.2;
Add ioconf entries to the pseudo-devices.

branches: 1.17.2; 1.17.4; 1.17.8;
NPF: partially rewrite the connection tracking mechanism:
- Separate the tracking interface from the storage (state table)
and thus prepare to use a new data structure for the storage.
- Fix some race conditions in NAT association logic.

branches: 1.16.2;
NPF: add support for specifying the interfaces before they are attached.
If an interface is or gets detached, all associated rules and connections
will be deactivated (it might be useful to have an option to invalidate
the associated connections). Once the interface is reattached they will
become active.

Bump NPF_VERSION.

NPF: G/C n-code in favour of BPF byte-code. Delete lots of code, mmm!

- Convert NPF to use BPF byte-code by default. Compile BPF byte-code in
npfctl(8) and generate separate marks to describe the filter criteria.
- Rewrite 'npfctl show' functionality and fix some of the bugs.
- npftest: add a test for BPF COP.
- Bump NPF_VERSION.

branches: 1.13.2;
- NPF connection tracking: rework synchronisation on tracking disable/enable
points and document it. Split the worker thread into a separate module
with an interface, so it could be re-used for other tasks.
- Replace ALG list with arrays and thus hit fewer cache lines.
- Misc bug fixes.

NPF:
- Implement dynamic NPF rules. Controlled through npf(3) library of via
npfctl rule command. A rule can be removed using a unique identifier,
returned on addition, or using a key which is SHA1 hash of the rule.
Adjust npftest and add a regression test.
- Improvements to rule inspection mechanism.
- Initial BPF support as an alternative to n-code.
- Minor fixes; bump the version.

Implement dynamic NPF extensions interface. An extension consists of
dynamically loaded module (.so) supplementing npfctl(8) and a kernel
module. Move normalisation and logging functionality into their own
extensions. More improvements to come.

branches: 1.10.2;
- Extend npftest: add ruleset inspection testing from the config generated
by npfctl debug functionality. Auto-create npftest interfaces for this.
- NPF sessions: combine protocol and interface into a separate substructure,
share between the entries and thus fix the handling of them. Constify.
- npftest: add regression tests for NAT policies.
- npf_build_nat: simplify and fix bi-NAT regression.
- Bump yacc stack size for npfctl.

branches: 1.9.2;
- Split NPF rule procedure code into a separate module (no functional changes).
- Simplify some code, add more comments, some asserts.
- G/C unused rule hook code.

branches: 1.8.2;
- Rework and improve TCP state tracking.
- Fix regressions after IPv6 patch merge.

Note: npfctl(8) rework will come soon.

Fix breakage caused by last commit. "npf_alg_icmp.c" is supposed to be
its own kernel module.

Define "INET6" to get IPv6 support.

Update list of source files.

branches: 1.4.6; 1.4.10;
NPF checkpoint:
- Add support for session saving/restoring.
- Add packet logging support (can tcpdump a pseudo-interface).
- Support reload without flushing of sessions; rework some locking.
- Revisit session mangement, replace linking with npf_sentry_t entries.
- Add some counters for statistics, using percpu(9).
- Add IP_DF flag cleansing.
- Fix various bugs; misc clean-up.

NPF checkpoint:
- Add proper TCP state tracking as described in Guido van Rooij paper,
plus handle TCP Window Scaling option.
- Completely rework npf_cache_t, reduce granularity, simplify code.
- Add npf_addr_t as an abstraction, amend session handling code, as well
as NAT code et al, to use it. Now design is prepared for IPv6 support.
- Handle IPv4 fragments i.e. perform packet reassembly.
- Add support for IPv4 ID randomization and minimum TTL enforcement.
- Add support for TCP MSS "clamping".
- Random bits for IPv6. Various fixes and clean-up.

branches: 1.2.2; 1.2.4;
NPF checkpoint:
- Add support for bi-directional NAT and redirection / port forwarding.
- Finish filtering on ICMP type/code and add filtering on TCP flags.
- Add support for TCP reset (RST) or ICMP destination unreachable on block.
- Fix a bunch of bugs; misc cleanup.

Import NPF - a packet filter. Some features:

- Designed to be fully MP-safe and highly efficient.

- Tables/IP sets (hash or red-black tree) for high performance lookups.

- Stateful filtering and Network Address Port Translation (NAPT).
Framework for application level gateways (ALGs).

- Packet inspection engine called n-code processor - inspired by BPF -
supporting generic RISC-like and specific CISC-like instructions for
common patterns (e.g. IPv4 address matching). See npf_ncode(9) manual.

- Convenient userland utility npfctl(8) with npf.conf(8).

NOTE: This is not yet a fully capable alternative to PF or IPFilter.
Further work (support for binat/rdr, return-rst/return-icmp, common ALGs,
state saving/restoring, logging, etc) is in progress.

Thanks a lot to Matt Thomas for various useful comments and code review.
Aye by: board@

branches: 1.1.2; 1.1.18;
Add ioconf entries to the pseudo-devices.