Home | History | Annotate | Line # | Download | only in system
      1      1.1  christos <!--
      2      1.1  christos Copyright (C) Internet Systems Consortium, Inc. ("ISC")
      3      1.1  christos 
      4      1.1  christos SPDX-License-Identifier: MPL-2.0
      5      1.1  christos 
      6      1.1  christos This Source Code Form is subject to the terms of the Mozilla Public
      7      1.1  christos License, v. 2.0.  If a copy of the MPL was not distributed with this
      8      1.1  christos file, you can obtain one at https://mozilla.org/MPL/2.0/.
      9      1.1  christos 
     10      1.1  christos See the COPYRIGHT file distributed with this work for additional
     11      1.1  christos information regarding copyright ownership.
     12      1.1  christos -->
     13      1.1  christos 
     14      1.1  christos # BIND9 System Test Cookbook
     15      1.1  christos 
     16      1.1  christos Task-oriented recipes for the system test framework.  Each recipe is
     17      1.1  christos self-contained and shows working code that can be copied and adapted.
     18      1.1  christos For concepts and reference documentation (build setup, fixtures, runner
     19      1.1  christos internals), see [README.md](README.md).
     20      1.1  christos 
     21      1.1  christos Prerequisite for all recipes: a built tree and the test network interfaces
     22      1.1  christos configured  see "Running the Tests" in the README.
     23      1.1  christos 
     24      1.1  christos 
     25      1.1  christos ## Iterate on a single test
     26      1.1  christos 
     27      1.1  christos The basic loop while developing a test:
     28      1.1  christos 
     29      1.1  christos ```sh
     30      1.1  christos cd bin/tests/system
     31      1.1  christos pytest mytest                                     # whole directory
     32      1.1  christos pytest mytest/tests_mytestmod.py::test_one_case   # one test function
     33      1.1  christos ```
     34      1.1  christos 
     35      1.1  christos When a test fails, its temporary directory is kept and a stable symlink to it
     36      1.1  christos is created (e.g. `mytest-mytestmod` for `mytest/tests_mytestmod.py`).  Look
     37      1.1  christos there for `pytest.log.txt` (the test's own log) and `ns*/named.run` (each
     38      1.1  christos server's debug-level log).  Pass `--noclean` to keep the directory even on
     39      1.1  christos success.
     40      1.1  christos 
     41      1.1  christos To poke at the servers interactively, pause the test at any point by
     42      1.1  christos inserting a breakpoint, or start pdb right away with `--trace`; to get a pdb
     43      1.1  christos prompt automatically when a test fails, use `--pdb`.  In all cases the
     44      1.1  christos servers keep running while the test is paused, so their state can be
     45      1.1  christos inspected:
     46      1.1  christos 
     47      1.1  christos ```sh
     48      1.1  christos pytest mytest --trace   # pdb at the start of each test
     49      1.1  christos pytest mytest --pdb     # pdb when a test fails
     50      1.1  christos ```
     51      1.1  christos 
     52      1.1  christos Timing-sensitive bugs hide in system tests: before declaring a test done, run
     53      1.1  christos it several times (and ideally once under parallel load, `pytest -n auto`).
     54      1.1  christos A test that flakes is treated as a bug in the test.
     55      1.1  christos 
     56      1.1  christos 
     57      1.1  christos ## Add a new system test directory
     58      1.1  christos 
     59      1.1  christos This skeleton sets up one authoritative server and queries it.  Pick a name
     60      1.1  christos that starts with a letter and uses underscores as word separators (hyphens
     61      1.1  christos are not allowed), and create:
     62      1.1  christos 
     63      1.1  christos ```
     64      1.1  christos demo/
     65      1.1  christos  ns1/
     66      1.1  christos     named.conf.j2
     67      1.1  christos     example.db
     68      1.1  christos  tests_demo.py
     69      1.1  christos ```
     70      1.1  christos 
     71      1.1  christos `demo/ns1/named.conf.j2`  the config template; the runner renders it to
     72  1.1.1.2  christos `named.conf` at setup time, filling in the assigned ports:
     73      1.1  christos 
     74      1.1  christos ```jinja
     75      1.1  christos options {
     76  1.1.1.2  christos 	{% include_indented "_common/options.conf.j2" %}
     77  1.1.1.2  christos 	recursion no;
     78  1.1.1.2  christos 	dnssec-validation no;
     79      1.1  christos };
     80      1.1  christos 
     81      1.1  christos {% include "_common/controls.conf.j2" %}
     82      1.1  christos 
     83      1.1  christos zone "example" {
     84  1.1.1.2  christos 	type primary;
     85  1.1.1.2  christos 	file "example.db";
     86      1.1  christos };
     87      1.1  christos ```
     88      1.1  christos 
     89  1.1.1.2  christos The `_common/options.conf.j2` include expands to the standard plumbing
     90  1.1.1.2  christos (configuring interfaces, source addresses, port and pid file).  Write only the
     91  1.1.1.2  christos options your test actually cares about below the include.  When the standard
     92  1.1.1.2  christos block doesn't fit, compose the pieces from `_common/options/` instead and put
     93  1.1.1.2  christos anything nonstandard inline.
     94  1.1.1.2  christos 
     95  1.1.1.2  christos The plumbing templates are built on the `ns` variable: templates inside an
     96  1.1.1.2  christos `nsN`/`ansN` subdirectory automatically get one describing that server, so a
     97  1.1.1.2  christos config never hardcodes its own address  `@ns.ip@` renders to 10.53.0.1 in
     98  1.1.1.2  christos ns1, 10.53.0.2 in ns2, ..., and `@ns.ip6@` to the matching
     99  1.1.1.2  christos fd92:7065:b8e:ffff::N address.  Use it wherever a config refers to the
    100  1.1.1.2  christos instance's own address; references to *other* servers stay literal.
    101  1.1.1.2  christos 
    102      1.1  christos The `_common/controls.conf.j2` include sets up the rndc control channel, so
    103  1.1.1.2  christos the test (and the runner's shutdown sequence) can use `rndc`.  A resolver
    104  1.1.1.2  christos instance would also include `_common/root.hint.conf` to get the standard
    105  1.1.1.2  christos root hints (ns1 is the root server by convention).  Inside an indented
    106  1.1.1.2  christos section such as a view statement, use `{% include_indented "..." %}` instead
    107  1.1.1.2  christos of `{% include %}`  it aligns the inserted block with the tag's own
    108  1.1.1.2  christos indentation.
    109      1.1  christos 
    110      1.1  christos `demo/ns1/example.db`  a plain zone file:
    111      1.1  christos 
    112      1.1  christos ```
    113      1.1  christos $TTL 300
    114      1.1  christos example.        IN SOA  ns1.example. hostmaster.example. 1 600 600 1200 600
    115      1.1  christos example.        NS      ns1.example.
    116      1.1  christos ns1.example.    A       10.53.0.1
    117      1.1  christos a.example.      A       10.0.0.1
    118      1.1  christos ```
    119      1.1  christos 
    120      1.1  christos `demo/tests_demo.py`  the test module:
    121      1.1  christos 
    122      1.1  christos ```python
    123      1.1  christos import isctest
    124      1.1  christos 
    125      1.1  christos 
    126      1.1  christos def test_a_record(ns1):
    127      1.1  christos     msg = isctest.query.create("a.example.", "A")
    128      1.1  christos     response = isctest.query.udp(msg, ns1.ip)
    129      1.1  christos     isctest.check.noerror(response)
    130      1.1  christos     assert str(response.answer[0][0]) == "10.0.0.1"
    131      1.1  christos ```
    132      1.1  christos 
    133      1.1  christos That's it  run it with `pytest demo`.  The pytest runner discovers new
    134      1.1  christos directories automatically; to also include the test in `make check` runs,
    135      1.1  christos add it to `TESTS` in `Makefile.am`.
    136      1.1  christos 
    137      1.1  christos Notes:
    138      1.1  christos 
    139      1.1  christos - The `ns1` fixture is the started server (an
    140      1.1  christos   `isctest.instance.NamedInstance`); `ns1.ip` is 10.53.0.1.  Servers in
    141      1.1  christos   `nsN`/`ansN` subdirectories are started automatically.
    142      1.1  christos - If your test writes files beyond the usual ones (e.g. `dig.out.*`,
    143      1.1  christos   `dsset-*`), declare them as a module-level marker 
    144      1.1  christos   `pytestmark = pytest.mark.extra_artifacts(["dig.out.*"])`  or the
    145      1.1  christos   artifact check at teardown will fail.  Most real test modules carry one.
    146      1.1  christos 
    147      1.1  christos 
    148      1.1  christos ## Set up zones in bootstrap()
    149      1.1  christos 
    150      1.1  christos A module-level `bootstrap()` function runs before the config templates are
    151      1.1  christos rendered and before the servers start.  This is where zone files and DNSSEC
    152      1.1  christos keys can be generated; whatever dict it returns becomes template data.
    153      1.1  christos 
    154      1.1  christos The simplest case is a plain unsigned zone  use this instead of writing
    155      1.1  christos the same boilerplate zone file by hand in every `nsN` directory:
    156      1.1  christos 
    157      1.1  christos ```python
    158      1.1  christos from isctest.template import NS1, zones
    159      1.1  christos from isctest.zone import Zone
    160      1.1  christos 
    161      1.1  christos 
    162      1.1  christos def bootstrap():
    163      1.1  christos     zone = Zone("example", NS1)    # signed=False is the default
    164      1.1  christos     zone.configure()               # render the zone file
    165      1.1  christos     return {"zones": zones([zone])}
    166      1.1  christos ```
    167      1.1  christos 
    168      1.1  christos `Zone.configure()` renders the zone file from
    169      1.1  christos `_common/zones/template.db.j2.manual`, a generic zone with SOA, NS, and a
    170      1.1  christos few test records.  To provide your own content, drop a template named after
    171      1.1  christos the zone file  `ns1/zones/<name>.db.j2.manual`  and `configure()` picks
    172      1.1  christos it up automatically instead of the generic one.  (The `.j2.manual` suffix
    173      1.1  christos keeps the file out of the runner's automatic template pass, which renders
    174      1.1  christos every plain `*.j2` at setup time; here `bootstrap()` renders it instead.)
    175      1.1  christos 
    176      1.1  christos For DNSSEC-signed zones, pass `signed=True` and make sure to configure the
    177      1.1  christos `trust_anchors`:
    178      1.1  christos 
    179      1.1  christos ```python
    180      1.1  christos from isctest.template import NS1, NS2, zones
    181      1.1  christos from isctest.zone import Zone, configure_root
    182      1.1  christos 
    183      1.1  christos 
    184      1.1  christos def bootstrap():
    185      1.1  christos     zone = Zone("signed-example", NS2, signed=True)
    186      1.1  christos     zone.configure()           # keys, zone file, dnssec-signzone
    187      1.1  christos 
    188      1.1  christos     # root zone on ns1, delegating to (and providing DS records for)
    189      1.1  christos     # the zone above
    190      1.1  christos     root = configure_root([zone])
    191      1.1  christos 
    192      1.1  christos     return {
    193      1.1  christos         "trust_anchors": root.trust_anchors(),
    194      1.1  christos         "zones": zones([root, zone]),
    195      1.1  christos     }
    196      1.1  christos ```
    197      1.1  christos 
    198      1.1  christos With `signed=True`, `configure()` also generates a KSK+ZSK with
    199      1.1  christos `dnssec-keygen` and signs the rendered zone with `dnssec-signzone`.  To
    200      1.1  christos sign a hand-written zone file checked into git (no rendering at all), put
    201      1.1  christos it at `ns2/zones/<name>.db` and drive the steps yourself:
    202      1.1  christos 
    203      1.1  christos ```python
    204      1.1  christos def bootstrap():
    205      1.1  christos     zone = Zone("signed-example", NS2, signed=True)
    206      1.1  christos     zone.add_keys()
    207      1.1  christos     zone.sign("-3 ABCD")       # extra dnssec-signzone args, e.g. NSEC3
    208      1.1  christos 
    209      1.1  christos     root = configure_root([zone])
    210      1.1  christos     return {
    211      1.1  christos         "trust_anchors": root.trust_anchors(),
    212      1.1  christos         "zones": zones([root, zone]),
    213      1.1  christos     }
    214      1.1  christos ```
    215      1.1  christos 
    216      1.1  christos On the template side, the `_common` includes consume the returned data: the
    217      1.1  christos authoritative server's `named.conf.j2` ends with
    218      1.1  christos 
    219      1.1  christos ```jinja
    220      1.1  christos {% include "_common/controls.conf.j2" %}
    221      1.1  christos {% include "_common/zones.conf.j2" %}
    222      1.1  christos ```
    223      1.1  christos 
    224      1.1  christos (`zones.conf.j2` emits a `zone` block for every zone whose nameserver
    225      1.1  christos matches the rendering server) and the validating resolver's with
    226      1.1  christos 
    227      1.1  christos ```jinja
    228      1.1  christos {% include "_common/controls.conf.j2" %}
    229      1.1  christos {% include "_common/trusted.conf.j2" %}
    230      1.1  christos {% include "_common/root.hint.conf" %}
    231      1.1  christos ```
    232      1.1  christos 
    233      1.1  christos (`trusted.conf.j2` emits `trust-anchors` from the `trust_anchors` data, and
    234      1.1  christos `root.hint.conf` points the resolver at ns1 for the root zone).
    235      1.1  christos 
    236      1.1  christos The `dnssec_py` directory is the canonical example of this pattern,
    237      1.1  christos including several modules with different zone setups sharing one directory.
    238      1.1  christos Declare the generated files as artifacts:
    239      1.1  christos 
    240      1.1  christos ```python
    241      1.1  christos pytestmark = pytest.mark.extra_artifacts(
    242      1.1  christos     [
    243      1.1  christos         "ns*/dsset-*",
    244      1.1  christos         "ns*/trusted.conf",
    245      1.1  christos         "ns*/zones/*.db",
    246      1.1  christos         "ns*/zones/*.db.signed",
    247      1.1  christos     ]
    248      1.1  christos )
    249      1.1  christos ```
    250      1.1  christos 
    251      1.1  christos 
    252      1.1  christos ## Drive named and watch its logs
    253      1.1  christos 
    254      1.1  christos The `NamedInstance` fixtures (`ns1` ... `ns11`) control the running servers.
    255      1.1  christos The cardinal rule: never `time.sleep()` waiting for the server  watch the
    256      1.1  christos log for the line that proves the event happened.
    257      1.1  christos 
    258      1.1  christos Wait for a log line caused by an action  enter the watcher *before*
    259      1.1  christos triggering the action, so the line cannot slip past unobserved:
    260      1.1  christos 
    261      1.1  christos ```python
    262      1.1  christos def test_cache_flush(ns4):
    263      1.1  christos     with ns4.watch_log_from_here() as watcher:
    264      1.1  christos         ns4.rndc("flush")
    265      1.1  christos         watcher.wait_for_line("flushing caches in all views succeeded")
    266      1.1  christos ```
    267      1.1  christos 
    268      1.1  christos `wait_for_line()` accepts a string, a compiled regex, or a list of either
    269      1.1  christos (any match wins) and returns the `re.Match`.  `wait_for_all([...])` waits
    270      1.1  christos until every pattern has appeared, `wait_for_sequence([...])` requires them
    271      1.1  christos in order.  To match lines that may already have been logged (e.g. during
    272      1.1  christos startup), use `watch_log_from_start()` instead.
    273      1.1  christos 
    274      1.1  christos Reconfigure a server mid-test by re-rendering its config template with new
    275      1.1  christos data, then reloading.  `reconfigure()` and `reload()` send the rndc command
    276      1.1  christos and wait for the corresponding completion line in the log:
    277      1.1  christos 
    278      1.1  christos ```python
    279      1.1  christos def test_with_new_config(ns1, templates):
    280      1.1  christos     templates.render("ns1/named.conf", {"flag": True})
    281      1.1  christos     ns1.reconfigure()
    282      1.1  christos ```
    283      1.1  christos 
    284      1.1  christos Dynamic updates go through `nsupdate`:
    285      1.1  christos 
    286      1.1  christos ```python
    287      1.1  christos import dns.update
    288      1.1  christos 
    289      1.1  christos def test_update(ns1):
    290      1.1  christos     update = dns.update.UpdateMessage("example.")
    291      1.1  christos     update.add("added.example.", 300, "A", "10.0.0.2")
    292      1.1  christos     ns1.nsupdate(update)
    293      1.1  christos ```
    294      1.1  christos 
    295      1.1  christos Send queries and check the responses with `isctest.query` and
    296      1.1  christos `isctest.check`:
    297      1.1  christos 
    298      1.1  christos ```python
    299      1.1  christos msg = isctest.query.create("a.example.", "A")
    300      1.1  christos response = isctest.query.udp(msg, ns1.ip)     # or .tcp(), .tls()
    301      1.1  christos isctest.check.noerror(response)               # rcode checks: nxdomain,
    302      1.1  christos                                               #  servfail, refused, ...
    303      1.1  christos isctest.check.adflag(response)                # flag checks: rdflag,
    304      1.1  christos                                               #  noraflag, ...
    305      1.1  christos isctest.check.same_answer(response, expected) # compare two responses
    306      1.1  christos ```
    307      1.1  christos 
    308      1.1  christos To wait until a zone transfer has happened, poll the SOA serial:
    309      1.1  christos 
    310      1.1  christos ```python
    311      1.1  christos isctest.query.wait_for_serial(ns2.ip, "example.", expected_serial)
    312      1.1  christos ```
    313      1.1  christos 
    314      1.1  christos For grepping a file the server produced (dumps, statistics), use
    315      1.1  christos `isctest.text`:
    316      1.1  christos 
    317      1.1  christos ```python
    318      1.1  christos from re import compile as Re
    319      1.1  christos 
    320      1.1  christos ns4.rndc("dumpdb -all")
    321      1.1  christos dump = isctest.text.TextFile("ns4/named_dump.db")
    322      1.1  christos assert dump.grep(Re(r"^a\.example\..*10\.0\.0\.1"))
    323      1.1  christos ```
    324