Home | History | Annotate | Line # | Download | only in system
COOKBOOK.md revision 1.1
      1  1.1  christos <!--
      2  1.1  christos Copyright (C) Internet Systems Consortium, Inc. ("ISC")
      3  1.1  christos 
      4  1.1  christos SPDX-License-Identifier: MPL-2.0
      5  1.1  christos 
      6  1.1  christos This Source Code Form is subject to the terms of the Mozilla Public
      7  1.1  christos License, v. 2.0.  If a copy of the MPL was not distributed with this
      8  1.1  christos file, you can obtain one at https://mozilla.org/MPL/2.0/.
      9  1.1  christos 
     10  1.1  christos See the COPYRIGHT file distributed with this work for additional
     11  1.1  christos information regarding copyright ownership.
     12  1.1  christos -->
     13  1.1  christos 
     14  1.1  christos # BIND9 System Test Cookbook
     15  1.1  christos 
     16  1.1  christos Task-oriented recipes for the system test framework.  Each recipe is
     17  1.1  christos self-contained and shows working code that can be copied and adapted.
     18  1.1  christos For concepts and reference documentation (build setup, fixtures, runner
     19  1.1  christos internals), see [README.md](README.md).
     20  1.1  christos 
     21  1.1  christos Prerequisite for all recipes: a built tree and the test network interfaces
     22  1.1  christos configured  see "Running the Tests" in the README.
     23  1.1  christos 
     24  1.1  christos 
     25  1.1  christos ## Iterate on a single test
     26  1.1  christos 
     27  1.1  christos The basic loop while developing a test:
     28  1.1  christos 
     29  1.1  christos ```sh
     30  1.1  christos cd bin/tests/system
     31  1.1  christos pytest mytest                                     # whole directory
     32  1.1  christos pytest mytest/tests_mytestmod.py::test_one_case   # one test function
     33  1.1  christos ```
     34  1.1  christos 
     35  1.1  christos When a test fails, its temporary directory is kept and a stable symlink to it
     36  1.1  christos is created (e.g. `mytest-mytestmod` for `mytest/tests_mytestmod.py`).  Look
     37  1.1  christos there for `pytest.log.txt` (the test's own log) and `ns*/named.run` (each
     38  1.1  christos server's debug-level log).  Pass `--noclean` to keep the directory even on
     39  1.1  christos success.
     40  1.1  christos 
     41  1.1  christos To poke at the servers interactively, pause the test at any point by
     42  1.1  christos inserting a breakpoint, or start pdb right away with `--trace`; to get a pdb
     43  1.1  christos prompt automatically when a test fails, use `--pdb`.  In all cases the
     44  1.1  christos servers keep running while the test is paused, so their state can be
     45  1.1  christos inspected:
     46  1.1  christos 
     47  1.1  christos ```sh
     48  1.1  christos pytest mytest --trace   # pdb at the start of each test
     49  1.1  christos pytest mytest --pdb     # pdb when a test fails
     50  1.1  christos ```
     51  1.1  christos 
     52  1.1  christos Timing-sensitive bugs hide in system tests: before declaring a test done, run
     53  1.1  christos it several times (and ideally once under parallel load, `pytest -n auto`).
     54  1.1  christos A test that flakes is treated as a bug in the test.
     55  1.1  christos 
     56  1.1  christos 
     57  1.1  christos ## Add a new system test directory
     58  1.1  christos 
     59  1.1  christos This skeleton sets up one authoritative server and queries it.  Pick a name
     60  1.1  christos that starts with a letter and uses underscores as word separators (hyphens
     61  1.1  christos are not allowed), and create:
     62  1.1  christos 
     63  1.1  christos ```
     64  1.1  christos demo/
     65  1.1  christos  ns1/
     66  1.1  christos     named.conf.j2
     67  1.1  christos     example.db
     68  1.1  christos  tests_demo.py
     69  1.1  christos ```
     70  1.1  christos 
     71  1.1  christos `demo/ns1/named.conf.j2`  the config template; the runner renders it to
     72  1.1  christos `named.conf` at setup time, filling in the assigned ports.  Templates inside
     73  1.1  christos an `nsN`/`ansN` subdirectory also get an `ns` variable describing that
     74  1.1  christos server, so the config doesn't hardcode its own address (`@ns.ip@` renders to
     75  1.1  christos 10.53.0.1 in ns1, 10.53.0.2 in ns2, ...):
     76  1.1  christos 
     77  1.1  christos ```jinja
     78  1.1  christos options {
     79  1.1  christos     query-source address @ns.ip@;
     80  1.1  christos     notify-source @ns.ip@;
     81  1.1  christos     transfer-source @ns.ip@;
     82  1.1  christos     port @PORT@;
     83  1.1  christos     pid-file "named.pid";
     84  1.1  christos     listen-on { @ns.ip@; };
     85  1.1  christos     listen-on-v6 { none; };
     86  1.1  christos     recursion no;
     87  1.1  christos     dnssec-validation no;
     88  1.1  christos };
     89  1.1  christos 
     90  1.1  christos {% include "_common/controls.conf.j2" %}
     91  1.1  christos 
     92  1.1  christos zone "example" {
     93  1.1  christos     type primary;
     94  1.1  christos     file "example.db";
     95  1.1  christos };
     96  1.1  christos ```
     97  1.1  christos 
     98  1.1  christos The `_common/controls.conf.j2` include sets up the rndc control channel, so
     99  1.1  christos the test (and the runner's shutdown sequence) can use `rndc`.
    100  1.1  christos 
    101  1.1  christos `demo/ns1/example.db`  a plain zone file:
    102  1.1  christos 
    103  1.1  christos ```
    104  1.1  christos $TTL 300
    105  1.1  christos example.        IN SOA  ns1.example. hostmaster.example. 1 600 600 1200 600
    106  1.1  christos example.        NS      ns1.example.
    107  1.1  christos ns1.example.    A       10.53.0.1
    108  1.1  christos a.example.      A       10.0.0.1
    109  1.1  christos ```
    110  1.1  christos 
    111  1.1  christos `demo/tests_demo.py`  the test module:
    112  1.1  christos 
    113  1.1  christos ```python
    114  1.1  christos import isctest
    115  1.1  christos 
    116  1.1  christos 
    117  1.1  christos def test_a_record(ns1):
    118  1.1  christos     msg = isctest.query.create("a.example.", "A")
    119  1.1  christos     response = isctest.query.udp(msg, ns1.ip)
    120  1.1  christos     isctest.check.noerror(response)
    121  1.1  christos     assert str(response.answer[0][0]) == "10.0.0.1"
    122  1.1  christos ```
    123  1.1  christos 
    124  1.1  christos That's it  run it with `pytest demo`.  The pytest runner discovers new
    125  1.1  christos directories automatically; to also include the test in `make check` runs,
    126  1.1  christos add it to `TESTS` in `Makefile.am`.
    127  1.1  christos 
    128  1.1  christos Notes:
    129  1.1  christos 
    130  1.1  christos - The `ns1` fixture is the started server (an
    131  1.1  christos   `isctest.instance.NamedInstance`); `ns1.ip` is 10.53.0.1.  Servers in
    132  1.1  christos   `nsN`/`ansN` subdirectories are started automatically.
    133  1.1  christos - If your test writes files beyond the usual ones (e.g. `dig.out.*`,
    134  1.1  christos   `dsset-*`), declare them as a module-level marker 
    135  1.1  christos   `pytestmark = pytest.mark.extra_artifacts(["dig.out.*"])`  or the
    136  1.1  christos   artifact check at teardown will fail.  Most real test modules carry one.
    137  1.1  christos 
    138  1.1  christos 
    139  1.1  christos ## Set up zones in bootstrap()
    140  1.1  christos 
    141  1.1  christos A module-level `bootstrap()` function runs before the config templates are
    142  1.1  christos rendered and before the servers start.  This is where zone files and DNSSEC
    143  1.1  christos keys can be generated; whatever dict it returns becomes template data.
    144  1.1  christos 
    145  1.1  christos The simplest case is a plain unsigned zone  use this instead of writing
    146  1.1  christos the same boilerplate zone file by hand in every `nsN` directory:
    147  1.1  christos 
    148  1.1  christos ```python
    149  1.1  christos from isctest.template import NS1, zones
    150  1.1  christos from isctest.zone import Zone
    151  1.1  christos 
    152  1.1  christos 
    153  1.1  christos def bootstrap():
    154  1.1  christos     zone = Zone("example", NS1)    # signed=False is the default
    155  1.1  christos     zone.configure()               # render the zone file
    156  1.1  christos     return {"zones": zones([zone])}
    157  1.1  christos ```
    158  1.1  christos 
    159  1.1  christos `Zone.configure()` renders the zone file from
    160  1.1  christos `_common/zones/template.db.j2.manual`, a generic zone with SOA, NS, and a
    161  1.1  christos few test records.  To provide your own content, drop a template named after
    162  1.1  christos the zone file  `ns1/zones/<name>.db.j2.manual`  and `configure()` picks
    163  1.1  christos it up automatically instead of the generic one.  (The `.j2.manual` suffix
    164  1.1  christos keeps the file out of the runner's automatic template pass, which renders
    165  1.1  christos every plain `*.j2` at setup time; here `bootstrap()` renders it instead.)
    166  1.1  christos 
    167  1.1  christos For DNSSEC-signed zones, pass `signed=True` and make sure to configure the
    168  1.1  christos `trust_anchors`:
    169  1.1  christos 
    170  1.1  christos ```python
    171  1.1  christos from isctest.template import NS1, NS2, zones
    172  1.1  christos from isctest.zone import Zone, configure_root
    173  1.1  christos 
    174  1.1  christos 
    175  1.1  christos def bootstrap():
    176  1.1  christos     zone = Zone("signed-example", NS2, signed=True)
    177  1.1  christos     zone.configure()           # keys, zone file, dnssec-signzone
    178  1.1  christos 
    179  1.1  christos     # root zone on ns1, delegating to (and providing DS records for)
    180  1.1  christos     # the zone above
    181  1.1  christos     root = configure_root([zone])
    182  1.1  christos 
    183  1.1  christos     return {
    184  1.1  christos         "trust_anchors": root.trust_anchors(),
    185  1.1  christos         "zones": zones([root, zone]),
    186  1.1  christos     }
    187  1.1  christos ```
    188  1.1  christos 
    189  1.1  christos With `signed=True`, `configure()` also generates a KSK+ZSK with
    190  1.1  christos `dnssec-keygen` and signs the rendered zone with `dnssec-signzone`.  To
    191  1.1  christos sign a hand-written zone file checked into git (no rendering at all), put
    192  1.1  christos it at `ns2/zones/<name>.db` and drive the steps yourself:
    193  1.1  christos 
    194  1.1  christos ```python
    195  1.1  christos def bootstrap():
    196  1.1  christos     zone = Zone("signed-example", NS2, signed=True)
    197  1.1  christos     zone.add_keys()
    198  1.1  christos     zone.sign("-3 ABCD")       # extra dnssec-signzone args, e.g. NSEC3
    199  1.1  christos 
    200  1.1  christos     root = configure_root([zone])
    201  1.1  christos     return {
    202  1.1  christos         "trust_anchors": root.trust_anchors(),
    203  1.1  christos         "zones": zones([root, zone]),
    204  1.1  christos     }
    205  1.1  christos ```
    206  1.1  christos 
    207  1.1  christos On the template side, the `_common` includes consume the returned data: the
    208  1.1  christos authoritative server's `named.conf.j2` ends with
    209  1.1  christos 
    210  1.1  christos ```jinja
    211  1.1  christos {% include "_common/controls.conf.j2" %}
    212  1.1  christos {% include "_common/zones.conf.j2" %}
    213  1.1  christos ```
    214  1.1  christos 
    215  1.1  christos (`zones.conf.j2` emits a `zone` block for every zone whose nameserver
    216  1.1  christos matches the rendering server) and the validating resolver's with
    217  1.1  christos 
    218  1.1  christos ```jinja
    219  1.1  christos {% include "_common/controls.conf.j2" %}
    220  1.1  christos {% include "_common/trusted.conf.j2" %}
    221  1.1  christos {% include "_common/root.hint.conf" %}
    222  1.1  christos ```
    223  1.1  christos 
    224  1.1  christos (`trusted.conf.j2` emits `trust-anchors` from the `trust_anchors` data, and
    225  1.1  christos `root.hint.conf` points the resolver at ns1 for the root zone).
    226  1.1  christos 
    227  1.1  christos The `dnssec_py` directory is the canonical example of this pattern,
    228  1.1  christos including several modules with different zone setups sharing one directory.
    229  1.1  christos Declare the generated files as artifacts:
    230  1.1  christos 
    231  1.1  christos ```python
    232  1.1  christos pytestmark = pytest.mark.extra_artifacts(
    233  1.1  christos     [
    234  1.1  christos         "ns*/dsset-*",
    235  1.1  christos         "ns*/trusted.conf",
    236  1.1  christos         "ns*/zones/*.db",
    237  1.1  christos         "ns*/zones/*.db.signed",
    238  1.1  christos     ]
    239  1.1  christos )
    240  1.1  christos ```
    241  1.1  christos 
    242  1.1  christos 
    243  1.1  christos ## Drive named and watch its logs
    244  1.1  christos 
    245  1.1  christos The `NamedInstance` fixtures (`ns1` ... `ns11`) control the running servers.
    246  1.1  christos The cardinal rule: never `time.sleep()` waiting for the server  watch the
    247  1.1  christos log for the line that proves the event happened.
    248  1.1  christos 
    249  1.1  christos Wait for a log line caused by an action  enter the watcher *before*
    250  1.1  christos triggering the action, so the line cannot slip past unobserved:
    251  1.1  christos 
    252  1.1  christos ```python
    253  1.1  christos def test_cache_flush(ns4):
    254  1.1  christos     with ns4.watch_log_from_here() as watcher:
    255  1.1  christos         ns4.rndc("flush")
    256  1.1  christos         watcher.wait_for_line("flushing caches in all views succeeded")
    257  1.1  christos ```
    258  1.1  christos 
    259  1.1  christos `wait_for_line()` accepts a string, a compiled regex, or a list of either
    260  1.1  christos (any match wins) and returns the `re.Match`.  `wait_for_all([...])` waits
    261  1.1  christos until every pattern has appeared, `wait_for_sequence([...])` requires them
    262  1.1  christos in order.  To match lines that may already have been logged (e.g. during
    263  1.1  christos startup), use `watch_log_from_start()` instead.
    264  1.1  christos 
    265  1.1  christos Reconfigure a server mid-test by re-rendering its config template with new
    266  1.1  christos data, then reloading.  `reconfigure()` and `reload()` send the rndc command
    267  1.1  christos and wait for the corresponding completion line in the log:
    268  1.1  christos 
    269  1.1  christos ```python
    270  1.1  christos def test_with_new_config(ns1, templates):
    271  1.1  christos     templates.render("ns1/named.conf", {"flag": True})
    272  1.1  christos     ns1.reconfigure()
    273  1.1  christos ```
    274  1.1  christos 
    275  1.1  christos Dynamic updates go through `nsupdate`:
    276  1.1  christos 
    277  1.1  christos ```python
    278  1.1  christos import dns.update
    279  1.1  christos 
    280  1.1  christos def test_update(ns1):
    281  1.1  christos     update = dns.update.UpdateMessage("example.")
    282  1.1  christos     update.add("added.example.", 300, "A", "10.0.0.2")
    283  1.1  christos     ns1.nsupdate(update)
    284  1.1  christos ```
    285  1.1  christos 
    286  1.1  christos Send queries and check the responses with `isctest.query` and
    287  1.1  christos `isctest.check`:
    288  1.1  christos 
    289  1.1  christos ```python
    290  1.1  christos msg = isctest.query.create("a.example.", "A")
    291  1.1  christos response = isctest.query.udp(msg, ns1.ip)     # or .tcp(), .tls()
    292  1.1  christos isctest.check.noerror(response)               # rcode checks: nxdomain,
    293  1.1  christos                                               #  servfail, refused, ...
    294  1.1  christos isctest.check.adflag(response)                # flag checks: rdflag,
    295  1.1  christos                                               #  noraflag, ...
    296  1.1  christos isctest.check.same_answer(response, expected) # compare two responses
    297  1.1  christos ```
    298  1.1  christos 
    299  1.1  christos To wait until a zone transfer has happened, poll the SOA serial:
    300  1.1  christos 
    301  1.1  christos ```python
    302  1.1  christos isctest.query.wait_for_serial(ns2.ip, "example.", expected_serial)
    303  1.1  christos ```
    304  1.1  christos 
    305  1.1  christos For grepping a file the server produced (dumps, statistics), use
    306  1.1  christos `isctest.text`:
    307  1.1  christos 
    308  1.1  christos ```python
    309  1.1  christos from re import compile as Re
    310  1.1  christos 
    311  1.1  christos ns4.rndc("dumpdb -all")
    312  1.1  christos dump = isctest.text.TextFile("ns4/named_dump.db")
    313  1.1  christos assert dump.grep(Re(r"^a\.example\..*10\.0\.0\.1"))
    314  1.1  christos ```
    315