Remove in-kernel handling of Router Advertisements

This is much better handled by a user-land tool.
Proposed on tech-net here:
https://mail-index.netbsd.org/tech-net/2020/04/22/msg007766.html

Note that the ioctl SIOCGIFINFO_IN6 no longer sets flags. That now
needs to be done using the pre-existing SIOCSIFINFO_FLAGS ioctl.

Compat is fully provided where it makes sense, but trying to turn on
RA handling will obviously throw an error as it no longer exists.

Note that if you use IPv6 temporary addresses, this now needs to be
turned on in dhcpcd.conf(5) rather than in sysctl.conf(5).

Reduce duplicate codes

Introduce net_common.sh that is to share common functions used in tests
for networking. This commit commonizes extract_new_packets. Other duplicate
codes will be moved to the file in further commits.

branches: 1.2.2;
Add tests for RA

From s-yamaguchi@IIJ (with some tweaks by me)

Add tests for NDP

Tweak tests; increase the size of NS packets for the addition of a nonce

Provide more informative reports on failures

branches: 1.12.12;
Share rump_server start/stop and interface creation/destruction functions

The common functions store socks of rump_servers, interfaces of rump_servers
and buses that intefaces connect and allow to destroy them with common
functions without specifying which socks, interfaces and buses we should
destroy.

This change reduces lots of similar setup/cleanup codes.

Move HIJACKING definition to net_common.sh

Reduce duplicate codes

Introduce net_common.sh that is to share common functions used in tests
for networking. This commit commonizes extract_new_packets. Other duplicate
codes will be moved to the file in further commits.

Accept DEBUG environment variable

By doing so, we can easily turn DEBUG on/off without modifying
the ATF scripts.

Add tests for sysctl net.inet6.ip6.dad_count

From suzu-ken@IIJ

Ignore case in duplicated

Adjust for new ifconfig output

And use -o match to provide informative error messages.

+ -lrumpdev

branches: 1.4.2;
Disable another tentative state check

It's too ephemeral to check robustly.

Improve test stability

- Take a diff between packet dumps and use it for packet checking
- it's resistant against packet reorder
- Seep 2 sec to make sure a NS message is sent
- Disable tentative state check for now
- it's too ephemeral to check robustly

Fix cleanup

Add tests for NDP

tests: skip ndp_cache_state on qemu

ndp_rtm: Only ping once

Pointless doing 3 pings.
On a slow system, it's possible that many RTM_MISS messages could
overflow into the next test.

tests: ndp_rtm: Check correct source address in RTA_AUTHOR

Fixes PR kern/55074.

tests: check RTA_AUTHOR in messages

tests: fix ARP and NDP tests for RTM_* messages

While here add tests for RTM_MISS.

tests: fix test header name

Make a permanet neighbor cache to avoid sending an NS packet disturbing the test

A receiver of an ICMPv6 request packet creates a stale cache entry and it turns
into the delay state on replying the packet. After 5 second, the receiver sends
an NS packet as a reachability confirmation, which disturbs the test and causes
a unexpected result.

Should fix PR misc/54451

branches: 1.33.2;
tests: shorten the expire time of neighbor caches to reduce the runtime of the tests

tests: test state transitions of neighbor caches

branches: 1.31.2;
Avoid setting IP addresses of the same subnet on different interface

If we do so, there will remain one route that is of a preceding address, but
that behavior is not documented and may be changed in the future. Tests
shouldn't rely on such a unstable behavior.

branches: 1.30.2;

Fix the ndp_rtm test the same way the arp_rtm test was fixed:
1. get pid of bg process with $! not $?
2. expect a single message from route monitor, not two, after ndp -d
3. run atf_check just once to verify correct output, not once for each string

Enable to remove multiple ARP/NDP entries for one destination

The kernel can have multiple ARP/NDP entries which have an indentical
destination on different interfaces. This is normal and can be
reproduce easily by ping -I or ping6 -S. We should be able to remove
such entries.

arp -d <ip> and ndp -d <ip> are changed to fetch all ARP/NDP entries
and remove matched entries. So we can remove multiple entries
described above. This fetch all and selective removal behavior is
the same as arp <ip> and ndp <ip>; they also do fetch all entries
and show only matched entries.

Related to PR 51179

Restore ARP/NDP entries to route show and netstat -r

Requested by dyoung@ some time ago

Drop RTF_UP from a routing message of a deleted ARP/NDP entry

Improve backward compatibility of (fake) routing messages on adding an ARP/NDP entry

A message originally included only DST and GATEWAY. Restore it.

Fix usage of routing messages on arp -d and ndp -d

It didn't work as we expected; we should set RTA_GATEWAY not
RTA_IFP on RTM_GET to return an if_index and the kernel should
use it on RTM_DELETE.

Test implicit removals of ARP/NDP entries

One test case reproudces PR 51179.

Fix typo

Don't create a permanent L2 cache entry on adding an address to an interface

It was created to copy FreeBSD, however actually the cache isn't
necessary. Remove it to simplify the code and reduce the cost to
maintain it (e.g., keep a consistency with a corresponding local
route).

Add missing declarations for cleanup

Test routing messages emitted on operations of ARP/NDP entries

branches: 1.19.2;
Change the default value of DEBUG of stable tests to false

Provide a more robust regexp for time formats of 1day-ish

branches: 1.17.2;
Share rump_server start/stop and interface creation/destruction functions

The common functions store socks of rump_servers, interfaces of rump_servers
and buses that intefaces connect and allow to destroy them with common
functions without specifying which socks, interfaces and buses we should
destroy.

This change reduces lots of similar setup/cleanup codes.

Move HIJACKING definition to net_common.sh

Reduce duplicate codes

Introduce net_common.sh that is to share common functions used in tests
for networking. This commit commonizes extract_new_packets. Other duplicate
codes will be moved to the file in further commits.

Accept DEBUG environment variable

By doing so, we can easily turn DEBUG on/off without modifying
the ATF scripts.

+ -lrumpdev

branches: 1.12.2;
Make a bunch of test names self-descriptive

Adjust the tests for temp option that works now

See PR kern/50127

Separate nexthop caches from the routing table

By this change, nexthop caches (IP-MAC address pair) are not stored
in the routing table anymore. Instead nexthop caches are stored in
each network interface; we already have lltable/llentry data structure
for this purpose. This change also obsoletes the concept of cloning/cloned
routes. Cloned routes no longer exist while cloning routes still exist
with renamed to connected routes.

Noticeable changes are:
- Nexthop caches aren't listed in route show/netstat -r
- sysctl(NET_RT_DUMP) doesn't return them
- If RTF_LLDATA is specified, it returns nexthop caches
- Several definitions of routing flags and messages are removed
- RTF_CLONING, RTF_XRESOLVE, RTF_LLINFO, RTF_CLONED and RTM_RESOLVE
- RTF_CONNECTED is added
- It has the same value of RTF_CLONING for backward compatibility
- route's -xresolve, -[no]cloned and -llinfo options are removed
- -[no]cloning remains because it seems there are users
- -[no]connected is introduced and recommended
to be used instead of -[no]cloning
- route show/netstat -r drops some flags
- 'L' and 'c' are not seen anymore
- 'C' now indicates a connected route
- Gateway value of a route of an interface address is now not
a L2 address but "link#N" like a connected (cloning) route
- Proxy ARP: "arp -s ... pub" doesn't create a route

You can know details of behavior changes by seeing diffs under tests/.

Proposed on tech-net and tech-kern:
http://mail-index.netbsd.org/tech-net/2016/03/11/msg005701.html

Add tests on activating a new MAC address

Don't assign unused IP address

It sometimes creates an unexpected NDP cache.

Add tests for GC of neighbor caches

Make a test a bit easy

Accept just 24h of expiration time in addition to 24h - a few seconds.

Improve test stability

A test for ndp -c was sometimes failed because between the deletion
and the check NS/NA messages were exchanged and a NDP cache was
recreated unexpectedly. To provent this situation, we do ifconfig
shmif0 down of the peer before the test, so the test won't be
interfered by the messages.

Fix head and cleanup definitions for cache_expiration tests

Use rump.ping6 instead of ping6 with rumphijack(3)

Check the output of ndp -d strictly

Add tests for NDP

Remove in-kernel handling of Router Advertisements

This is much better handled by a user-land tool.
Proposed on tech-net here:
https://mail-index.netbsd.org/tech-net/2020/04/22/msg007766.html

Note that the ioctl SIOCGIFINFO_IN6 no longer sets flags. That now
needs to be done using the pre-existing SIOCSIFINFO_FLAGS ioctl.

Compat is fully provided where it makes sense, but trying to turn on
RA handling will obviously throw an error as it no longer exists.

Note that if you use IPv6 temporary addresses, this now needs to be
turned on in dhcpcd.conf(5) rather than in sysctl.conf(5).

tests: add tests for the validateion of net.inet6.ip6.temppltime

branches: 1.32.4; 1.32.6;
Make this test somewhat deterministic - far fewer races, and most
of what are left are "race for the bus" type - if we lose, we just
wait for the next one ... slower but still reliable.

There are two exceptions ... when starting more than one rtadvd
(on different routers) we expect to receive an RA from each, but
all that we can check is that we received the (at least) right number
of RAs. It is possible (though unlikely) that one router sent two
before another sent any, in which case we will not have the data we
expect, and a sub-test will fail.

Second, there is no way to know for sure that we have waited long
enough when we're waiting for data to expire - in systems with
correctly working clocks that actually measure time, this should not
be an issue, if data is due to expire in < 5 seconds, and we wait
5 seconds, and the data is still there, then that indicates a
failure, which should be detected. Unfortunately with QEMU testing
time just isn't that reliable. But fortunately, it is generally the
sleep which takes longer, while other timers run correctly, which is
the way that makes us happy...

While here lots of cleanups - everything from white space and
line wrapping, to removing superfluous quotes and adding some
(but probably not enough) that are not (though given the data is
all known here, lack of quotes will rarely hurt.)

Also take note of the fact that current rtadvd *cannot* delete its
pidfile, so waiting for that file to be removed is doomed to failure.
Do things in a way that works, rather than simply resorting to assassination.

Because we do a lot less "sleep and hope it is long enough" and more
"wait until it is observed to happen" the tests generally run in less
elapsed time than before (20% less has been observed.) But because we
"wait until it is observed to happen" rather than just "sleep and hope
it is long enough" sometimes things take longer (and when that happens,
we no longer fail). Up to 7% slower (overall) has been observed.
(Observations on an amd64 DomU, no idea yet as to what QEMU might observe.)

Let rtadvd not use syslog for logging

Thanks to christos@ now rtadvd can log via stderr instead of syslog
by -D option.

Address PR bin/52701

Kill rtadvd surely even if the tests fail in the middle

It may help PR bin/52701.

Purge all related L2 caches on removing a route

The change addresses situations similar to PR 51179.

Don't create a permanent L2 cache entry on adding an address to an interface

It was created to copy FreeBSD, however actually the cache isn't
necessary. Remove it to simplify the code and reduce the cost to
maintain it (e.g., keep a consistency with a corresponding local
route).

branches: 1.27.2;
Change the default value of DEBUG of stable tests to false

Provide a more robust regexp for time formats of 1day-ish

Add tests for expiration of default router and prefix entries

branches: 1.24.2;
Remove a check added wrongly

Add tests for net.inet6.ip6.prefer_tempaddr

Remove extra checks and cleanup

Cope with tentative state

Add a test case for IPv6 temporary address

Check autoconf flag

Fix typo

Restore multiple_routers_single_prefix_cleanup removed wrongly

Reduce unnecessary wait

Add a test case for exceeding the number of maximum prefixes

The test case pinpoints purge_detached.

Add tests for multiple routers with a single prefix

Fix the description of a test

Add tests for multiple routers

Unify common routines

Avoid using /var/run/rump.rtadvd.pid

Add a test case that deletes auto-configured addresses

Improve stability of the tests

- Do ifconfig -w 10 after ifconfig up
- Accept /1d0h0m..s/ in addition to /23h59m..s/ for expiration time
- Prevent new RA messages from coming after flushing entries

The changes should fix flapping of test results on babylon5.

Add tests for flushing prefix and default router entries

Share rump_server start/stop and interface creation/destruction functions

The common functions store socks of rump_servers, interfaces of rump_servers
and buses that intefaces connect and allow to destroy them with common
functions without specifying which socks, interfaces and buses we should
destroy.

This change reduces lots of similar setup/cleanup codes.

Accept DEBUG environment variable

By doing so, we can easily turn DEBUG on/off without modifying
the ATF scripts.

Make test names self-descriptive

+ -lrumpdev

branches: 1.2.2;
Fix up the header

Remove unnecessary shebang and add missing keyword expansion,
copyright and license.

Add tests for RA

From s-yamaguchi@IIJ (with some tweaks by me)