1 1.1 christos <!-- 2 1.1 christos Copyright (C) Internet Systems Consortium, Inc. ("ISC") 3 1.1 christos 4 1.1 christos SPDX-License-Identifier: MPL-2.0 5 1.1 christos 6 1.1 christos This Source Code Form is subject to the terms of the Mozilla Public 7 1.1 christos License, v. 2.0. If a copy of the MPL was not distributed with this 8 1.1 christos file, you can obtain one at https://mozilla.org/MPL/2.0/. 9 1.1 christos 10 1.1 christos See the COPYRIGHT file distributed with this work for additional 11 1.1 christos information regarding copyright ownership. 12 1.1 christos --> 13 1.1 christos 14 1.1 christos # BIND9 System Test Cookbook 15 1.1 christos 16 1.1 christos Task-oriented recipes for the system test framework. Each recipe is 17 1.1 christos self-contained and shows working code that can be copied and adapted. 18 1.1 christos For concepts and reference documentation (build setup, fixtures, runner 19 1.1 christos internals), see [README.md](README.md). 20 1.1 christos 21 1.1 christos Prerequisite for all recipes: a built tree and the test network interfaces 22 1.1 christos configured see "Running the Tests" in the README. 23 1.1 christos 24 1.1 christos 25 1.1 christos ## Iterate on a single test 26 1.1 christos 27 1.1 christos The basic loop while developing a test: 28 1.1 christos 29 1.1 christos ```sh 30 1.1 christos cd bin/tests/system 31 1.1 christos pytest mytest # whole directory 32 1.1 christos pytest mytest/tests_mytestmod.py::test_one_case # one test function 33 1.1 christos ``` 34 1.1 christos 35 1.1 christos When a test fails, its temporary directory is kept and a stable symlink to it 36 1.1 christos is created (e.g. `mytest-mytestmod` for `mytest/tests_mytestmod.py`). Look 37 1.1 christos there for `pytest.log.txt` (the test's own log) and `ns*/named.run` (each 38 1.1 christos server's debug-level log). Pass `--noclean` to keep the directory even on 39 1.1 christos success. 40 1.1 christos 41 1.1 christos To poke at the servers interactively, pause the test at any point by 42 1.1 christos inserting a breakpoint, or start pdb right away with `--trace`; to get a pdb 43 1.1 christos prompt automatically when a test fails, use `--pdb`. In all cases the 44 1.1 christos servers keep running while the test is paused, so their state can be 45 1.1 christos inspected: 46 1.1 christos 47 1.1 christos ```sh 48 1.1 christos pytest mytest --trace # pdb at the start of each test 49 1.1 christos pytest mytest --pdb # pdb when a test fails 50 1.1 christos ``` 51 1.1 christos 52 1.1 christos Timing-sensitive bugs hide in system tests: before declaring a test done, run 53 1.1 christos it several times (and ideally once under parallel load, `pytest -n auto`). 54 1.1 christos A test that flakes is treated as a bug in the test. 55 1.1 christos 56 1.1 christos 57 1.1 christos ## Add a new system test directory 58 1.1 christos 59 1.1 christos This skeleton sets up one authoritative server and queries it. Pick a name 60 1.1 christos that starts with a letter and uses underscores as word separators (hyphens 61 1.1 christos are not allowed), and create: 62 1.1 christos 63 1.1 christos ``` 64 1.1 christos demo/ 65 1.1 christos ns1/ 66 1.1 christos named.conf.j2 67 1.1 christos example.db 68 1.1 christos tests_demo.py 69 1.1 christos ``` 70 1.1 christos 71 1.1 christos `demo/ns1/named.conf.j2` the config template; the runner renders it to 72 1.1.1.2 christos `named.conf` at setup time, filling in the assigned ports: 73 1.1 christos 74 1.1 christos ```jinja 75 1.1 christos options { 76 1.1.1.2 christos {% include_indented "_common/options.conf.j2" %} 77 1.1.1.2 christos recursion no; 78 1.1.1.2 christos dnssec-validation no; 79 1.1 christos }; 80 1.1 christos 81 1.1 christos {% include "_common/controls.conf.j2" %} 82 1.1 christos 83 1.1 christos zone "example" { 84 1.1.1.2 christos type primary; 85 1.1.1.2 christos file "example.db"; 86 1.1 christos }; 87 1.1 christos ``` 88 1.1 christos 89 1.1.1.2 christos The `_common/options.conf.j2` include expands to the standard plumbing 90 1.1.1.2 christos (configuring interfaces, source addresses, port and pid file). Write only the 91 1.1.1.2 christos options your test actually cares about below the include. When the standard 92 1.1.1.2 christos block doesn't fit, compose the pieces from `_common/options/` instead and put 93 1.1.1.2 christos anything nonstandard inline. 94 1.1.1.2 christos 95 1.1.1.2 christos The plumbing templates are built on the `ns` variable: templates inside an 96 1.1.1.2 christos `nsN`/`ansN` subdirectory automatically get one describing that server, so a 97 1.1.1.2 christos config never hardcodes its own address `@ns.ip@` renders to 10.53.0.1 in 98 1.1.1.2 christos ns1, 10.53.0.2 in ns2, ..., and `@ns.ip6@` to the matching 99 1.1.1.2 christos fd92:7065:b8e:ffff::N address. Use it wherever a config refers to the 100 1.1.1.2 christos instance's own address; references to *other* servers stay literal. 101 1.1.1.2 christos 102 1.1 christos The `_common/controls.conf.j2` include sets up the rndc control channel, so 103 1.1.1.2 christos the test (and the runner's shutdown sequence) can use `rndc`. A resolver 104 1.1.1.2 christos instance would also include `_common/root.hint.conf` to get the standard 105 1.1.1.2 christos root hints (ns1 is the root server by convention). Inside an indented 106 1.1.1.2 christos section such as a view statement, use `{% include_indented "..." %}` instead 107 1.1.1.2 christos of `{% include %}` it aligns the inserted block with the tag's own 108 1.1.1.2 christos indentation. 109 1.1 christos 110 1.1 christos `demo/ns1/example.db` a plain zone file: 111 1.1 christos 112 1.1 christos ``` 113 1.1 christos $TTL 300 114 1.1 christos example. IN SOA ns1.example. hostmaster.example. 1 600 600 1200 600 115 1.1 christos example. NS ns1.example. 116 1.1 christos ns1.example. A 10.53.0.1 117 1.1 christos a.example. A 10.0.0.1 118 1.1 christos ``` 119 1.1 christos 120 1.1 christos `demo/tests_demo.py` the test module: 121 1.1 christos 122 1.1 christos ```python 123 1.1 christos import isctest 124 1.1 christos 125 1.1 christos 126 1.1 christos def test_a_record(ns1): 127 1.1 christos msg = isctest.query.create("a.example.", "A") 128 1.1 christos response = isctest.query.udp(msg, ns1.ip) 129 1.1 christos isctest.check.noerror(response) 130 1.1 christos assert str(response.answer[0][0]) == "10.0.0.1" 131 1.1 christos ``` 132 1.1 christos 133 1.1 christos That's it run it with `pytest demo`. The pytest runner discovers new 134 1.1 christos directories automatically; to also include the test in `make check` runs, 135 1.1 christos add it to `TESTS` in `Makefile.am`. 136 1.1 christos 137 1.1 christos Notes: 138 1.1 christos 139 1.1 christos - The `ns1` fixture is the started server (an 140 1.1 christos `isctest.instance.NamedInstance`); `ns1.ip` is 10.53.0.1. Servers in 141 1.1 christos `nsN`/`ansN` subdirectories are started automatically. 142 1.1 christos - If your test writes files beyond the usual ones (e.g. `dig.out.*`, 143 1.1 christos `dsset-*`), declare them as a module-level marker 144 1.1 christos `pytestmark = pytest.mark.extra_artifacts(["dig.out.*"])` or the 145 1.1 christos artifact check at teardown will fail. Most real test modules carry one. 146 1.1 christos 147 1.1 christos 148 1.1 christos ## Set up zones in bootstrap() 149 1.1 christos 150 1.1 christos A module-level `bootstrap()` function runs before the config templates are 151 1.1 christos rendered and before the servers start. This is where zone files and DNSSEC 152 1.1 christos keys can be generated; whatever dict it returns becomes template data. 153 1.1 christos 154 1.1 christos The simplest case is a plain unsigned zone use this instead of writing 155 1.1 christos the same boilerplate zone file by hand in every `nsN` directory: 156 1.1 christos 157 1.1 christos ```python 158 1.1 christos from isctest.template import NS1, zones 159 1.1 christos from isctest.zone import Zone 160 1.1 christos 161 1.1 christos 162 1.1 christos def bootstrap(): 163 1.1 christos zone = Zone("example", NS1) # signed=False is the default 164 1.1 christos zone.configure() # render the zone file 165 1.1 christos return {"zones": zones([zone])} 166 1.1 christos ``` 167 1.1 christos 168 1.1 christos `Zone.configure()` renders the zone file from 169 1.1 christos `_common/zones/template.db.j2.manual`, a generic zone with SOA, NS, and a 170 1.1 christos few test records. To provide your own content, drop a template named after 171 1.1 christos the zone file `ns1/zones/<name>.db.j2.manual` and `configure()` picks 172 1.1 christos it up automatically instead of the generic one. (The `.j2.manual` suffix 173 1.1 christos keeps the file out of the runner's automatic template pass, which renders 174 1.1 christos every plain `*.j2` at setup time; here `bootstrap()` renders it instead.) 175 1.1 christos 176 1.1 christos For DNSSEC-signed zones, pass `signed=True` and make sure to configure the 177 1.1 christos `trust_anchors`: 178 1.1 christos 179 1.1 christos ```python 180 1.1 christos from isctest.template import NS1, NS2, zones 181 1.1 christos from isctest.zone import Zone, configure_root 182 1.1 christos 183 1.1 christos 184 1.1 christos def bootstrap(): 185 1.1 christos zone = Zone("signed-example", NS2, signed=True) 186 1.1 christos zone.configure() # keys, zone file, dnssec-signzone 187 1.1 christos 188 1.1 christos # root zone on ns1, delegating to (and providing DS records for) 189 1.1 christos # the zone above 190 1.1 christos root = configure_root([zone]) 191 1.1 christos 192 1.1 christos return { 193 1.1 christos "trust_anchors": root.trust_anchors(), 194 1.1 christos "zones": zones([root, zone]), 195 1.1 christos } 196 1.1 christos ``` 197 1.1 christos 198 1.1 christos With `signed=True`, `configure()` also generates a KSK+ZSK with 199 1.1 christos `dnssec-keygen` and signs the rendered zone with `dnssec-signzone`. To 200 1.1 christos sign a hand-written zone file checked into git (no rendering at all), put 201 1.1 christos it at `ns2/zones/<name>.db` and drive the steps yourself: 202 1.1 christos 203 1.1 christos ```python 204 1.1 christos def bootstrap(): 205 1.1 christos zone = Zone("signed-example", NS2, signed=True) 206 1.1 christos zone.add_keys() 207 1.1 christos zone.sign("-3 ABCD") # extra dnssec-signzone args, e.g. NSEC3 208 1.1 christos 209 1.1 christos root = configure_root([zone]) 210 1.1 christos return { 211 1.1 christos "trust_anchors": root.trust_anchors(), 212 1.1 christos "zones": zones([root, zone]), 213 1.1 christos } 214 1.1 christos ``` 215 1.1 christos 216 1.1 christos On the template side, the `_common` includes consume the returned data: the 217 1.1 christos authoritative server's `named.conf.j2` ends with 218 1.1 christos 219 1.1 christos ```jinja 220 1.1 christos {% include "_common/controls.conf.j2" %} 221 1.1 christos {% include "_common/zones.conf.j2" %} 222 1.1 christos ``` 223 1.1 christos 224 1.1 christos (`zones.conf.j2` emits a `zone` block for every zone whose nameserver 225 1.1 christos matches the rendering server) and the validating resolver's with 226 1.1 christos 227 1.1 christos ```jinja 228 1.1 christos {% include "_common/controls.conf.j2" %} 229 1.1 christos {% include "_common/trusted.conf.j2" %} 230 1.1 christos {% include "_common/root.hint.conf" %} 231 1.1 christos ``` 232 1.1 christos 233 1.1 christos (`trusted.conf.j2` emits `trust-anchors` from the `trust_anchors` data, and 234 1.1 christos `root.hint.conf` points the resolver at ns1 for the root zone). 235 1.1 christos 236 1.1 christos The `dnssec_py` directory is the canonical example of this pattern, 237 1.1 christos including several modules with different zone setups sharing one directory. 238 1.1 christos Declare the generated files as artifacts: 239 1.1 christos 240 1.1 christos ```python 241 1.1 christos pytestmark = pytest.mark.extra_artifacts( 242 1.1 christos [ 243 1.1 christos "ns*/dsset-*", 244 1.1 christos "ns*/trusted.conf", 245 1.1 christos "ns*/zones/*.db", 246 1.1 christos "ns*/zones/*.db.signed", 247 1.1 christos ] 248 1.1 christos ) 249 1.1 christos ``` 250 1.1 christos 251 1.1 christos 252 1.1 christos ## Drive named and watch its logs 253 1.1 christos 254 1.1 christos The `NamedInstance` fixtures (`ns1` ... `ns11`) control the running servers. 255 1.1 christos The cardinal rule: never `time.sleep()` waiting for the server watch the 256 1.1 christos log for the line that proves the event happened. 257 1.1 christos 258 1.1 christos Wait for a log line caused by an action enter the watcher *before* 259 1.1 christos triggering the action, so the line cannot slip past unobserved: 260 1.1 christos 261 1.1 christos ```python 262 1.1 christos def test_cache_flush(ns4): 263 1.1 christos with ns4.watch_log_from_here() as watcher: 264 1.1 christos ns4.rndc("flush") 265 1.1 christos watcher.wait_for_line("flushing caches in all views succeeded") 266 1.1 christos ``` 267 1.1 christos 268 1.1 christos `wait_for_line()` accepts a string, a compiled regex, or a list of either 269 1.1 christos (any match wins) and returns the `re.Match`. `wait_for_all([...])` waits 270 1.1 christos until every pattern has appeared, `wait_for_sequence([...])` requires them 271 1.1 christos in order. To match lines that may already have been logged (e.g. during 272 1.1 christos startup), use `watch_log_from_start()` instead. 273 1.1 christos 274 1.1 christos Reconfigure a server mid-test by re-rendering its config template with new 275 1.1 christos data, then reloading. `reconfigure()` and `reload()` send the rndc command 276 1.1 christos and wait for the corresponding completion line in the log: 277 1.1 christos 278 1.1 christos ```python 279 1.1 christos def test_with_new_config(ns1, templates): 280 1.1 christos templates.render("ns1/named.conf", {"flag": True}) 281 1.1 christos ns1.reconfigure() 282 1.1 christos ``` 283 1.1 christos 284 1.1 christos Dynamic updates go through `nsupdate`: 285 1.1 christos 286 1.1 christos ```python 287 1.1 christos import dns.update 288 1.1 christos 289 1.1 christos def test_update(ns1): 290 1.1 christos update = dns.update.UpdateMessage("example.") 291 1.1 christos update.add("added.example.", 300, "A", "10.0.0.2") 292 1.1 christos ns1.nsupdate(update) 293 1.1 christos ``` 294 1.1 christos 295 1.1 christos Send queries and check the responses with `isctest.query` and 296 1.1 christos `isctest.check`: 297 1.1 christos 298 1.1 christos ```python 299 1.1 christos msg = isctest.query.create("a.example.", "A") 300 1.1 christos response = isctest.query.udp(msg, ns1.ip) # or .tcp(), .tls() 301 1.1 christos isctest.check.noerror(response) # rcode checks: nxdomain, 302 1.1 christos # servfail, refused, ... 303 1.1 christos isctest.check.adflag(response) # flag checks: rdflag, 304 1.1 christos # noraflag, ... 305 1.1 christos isctest.check.same_answer(response, expected) # compare two responses 306 1.1 christos ``` 307 1.1 christos 308 1.1 christos To wait until a zone transfer has happened, poll the SOA serial: 309 1.1 christos 310 1.1 christos ```python 311 1.1 christos isctest.query.wait_for_serial(ns2.ip, "example.", expected_serial) 312 1.1 christos ``` 313 1.1 christos 314 1.1 christos For grepping a file the server produced (dumps, statistics), use 315 1.1 christos `isctest.text`: 316 1.1 christos 317 1.1 christos ```python 318 1.1 christos from re import compile as Re 319 1.1 christos 320 1.1 christos ns4.rndc("dumpdb -all") 321 1.1 christos dump = isctest.text.TextFile("ns4/named_dump.db") 322 1.1 christos assert dump.grep(Re(r"^a\.example\..*10\.0\.0\.1")) 323 1.1 christos ``` 324