1 <!-- 2 Copyright (C) Internet Systems Consortium, Inc. ("ISC") 3 4 SPDX-License-Identifier: MPL-2.0 5 6 This Source Code Form is subject to the terms of the Mozilla Public 7 License, v. 2.0. If a copy of the MPL was not distributed with this 8 file, you can obtain one at https://mozilla.org/MPL/2.0/. 9 10 See the COPYRIGHT file distributed with this work for additional 11 information regarding copyright ownership. 12 --> 13 14 # BIND9 System Test Cookbook 15 16 Task-oriented recipes for the system test framework. Each recipe is 17 self-contained and shows working code that can be copied and adapted. 18 For concepts and reference documentation (build setup, fixtures, runner 19 internals), see [README.md](README.md). 20 21 Prerequisite for all recipes: a built tree and the test network interfaces 22 configured see "Running the Tests" in the README. 23 24 25 ## Iterate on a single test 26 27 The basic loop while developing a test: 28 29 ```sh 30 cd bin/tests/system 31 pytest mytest # whole directory 32 pytest mytest/tests_mytestmod.py::test_one_case # one test function 33 ``` 34 35 When a test fails, its temporary directory is kept and a stable symlink to it 36 is created (e.g. `mytest-mytestmod` for `mytest/tests_mytestmod.py`). Look 37 there for `pytest.log.txt` (the test's own log) and `ns*/named.run` (each 38 server's debug-level log). Pass `--noclean` to keep the directory even on 39 success. 40 41 To poke at the servers interactively, pause the test at any point by 42 inserting a breakpoint, or start pdb right away with `--trace`; to get a pdb 43 prompt automatically when a test fails, use `--pdb`. In all cases the 44 servers keep running while the test is paused, so their state can be 45 inspected: 46 47 ```sh 48 pytest mytest --trace # pdb at the start of each test 49 pytest mytest --pdb # pdb when a test fails 50 ``` 51 52 Timing-sensitive bugs hide in system tests: before declaring a test done, run 53 it several times (and ideally once under parallel load, `pytest -n auto`). 54 A test that flakes is treated as a bug in the test. 55 56 57 ## Add a new system test directory 58 59 This skeleton sets up one authoritative server and queries it. Pick a name 60 that starts with a letter and uses underscores as word separators (hyphens 61 are not allowed), and create: 62 63 ``` 64 demo/ 65 ns1/ 66 named.conf.j2 67 example.db 68 tests_demo.py 69 ``` 70 71 `demo/ns1/named.conf.j2` the config template; the runner renders it to 72 `named.conf` at setup time, filling in the assigned ports: 73 74 ```jinja 75 options { 76 {% include_indented "_common/options.conf.j2" %} 77 recursion no; 78 dnssec-validation no; 79 }; 80 81 {% include "_common/controls.conf.j2" %} 82 83 zone "example" { 84 type primary; 85 file "example.db"; 86 }; 87 ``` 88 89 The `_common/options.conf.j2` include expands to the standard plumbing 90 (configuring interfaces, source addresses, port and pid file). Write only the 91 options your test actually cares about below the include. When the standard 92 block doesn't fit, compose the pieces from `_common/options/` instead and put 93 anything nonstandard inline. 94 95 The plumbing templates are built on the `ns` variable: templates inside an 96 `nsN`/`ansN` subdirectory automatically get one describing that server, so a 97 config never hardcodes its own address `@ns.ip@` renders to 10.53.0.1 in 98 ns1, 10.53.0.2 in ns2, ..., and `@ns.ip6@` to the matching 99 fd92:7065:b8e:ffff::N address. Use it wherever a config refers to the 100 instance's own address; references to *other* servers stay literal. 101 102 The `_common/controls.conf.j2` include sets up the rndc control channel, so 103 the test (and the runner's shutdown sequence) can use `rndc`. A resolver 104 instance would also include `_common/root.hint.conf` to get the standard 105 root hints (ns1 is the root server by convention). Inside an indented 106 section such as a view statement, use `{% include_indented "..." %}` instead 107 of `{% include %}` it aligns the inserted block with the tag's own 108 indentation. 109 110 `demo/ns1/example.db` a plain zone file: 111 112 ``` 113 $TTL 300 114 example. IN SOA ns1.example. hostmaster.example. 1 600 600 1200 600 115 example. NS ns1.example. 116 ns1.example. A 10.53.0.1 117 a.example. A 10.0.0.1 118 ``` 119 120 `demo/tests_demo.py` the test module: 121 122 ```python 123 import isctest 124 125 126 def test_a_record(ns1): 127 msg = isctest.query.create("a.example.", "A") 128 response = isctest.query.udp(msg, ns1.ip) 129 isctest.check.noerror(response) 130 assert str(response.answer[0][0]) == "10.0.0.1" 131 ``` 132 133 That's it run it with `pytest demo`. The pytest runner discovers new 134 directories automatically; to also include the test in `make check` runs, 135 add it to `TESTS` in `Makefile.am`. 136 137 Notes: 138 139 - The `ns1` fixture is the started server (an 140 `isctest.instance.NamedInstance`); `ns1.ip` is 10.53.0.1. Servers in 141 `nsN`/`ansN` subdirectories are started automatically. 142 - If your test writes files beyond the usual ones (e.g. `dig.out.*`, 143 `dsset-*`), declare them as a module-level marker 144 `pytestmark = pytest.mark.extra_artifacts(["dig.out.*"])` or the 145 artifact check at teardown will fail. Most real test modules carry one. 146 147 148 ## Set up zones in bootstrap() 149 150 A module-level `bootstrap()` function runs before the config templates are 151 rendered and before the servers start. This is where zone files and DNSSEC 152 keys can be generated; whatever dict it returns becomes template data. 153 154 The simplest case is a plain unsigned zone use this instead of writing 155 the same boilerplate zone file by hand in every `nsN` directory: 156 157 ```python 158 from isctest.template import NS1, zones 159 from isctest.zone import Zone 160 161 162 def bootstrap(): 163 zone = Zone("example", NS1) # signed=False is the default 164 zone.configure() # render the zone file 165 return {"zones": zones([zone])} 166 ``` 167 168 `Zone.configure()` renders the zone file from 169 `_common/zones/template.db.j2.manual`, a generic zone with SOA, NS, and a 170 few test records. To provide your own content, drop a template named after 171 the zone file `ns1/zones/<name>.db.j2.manual` and `configure()` picks 172 it up automatically instead of the generic one. (The `.j2.manual` suffix 173 keeps the file out of the runner's automatic template pass, which renders 174 every plain `*.j2` at setup time; here `bootstrap()` renders it instead.) 175 176 For DNSSEC-signed zones, pass `signed=True` and make sure to configure the 177 `trust_anchors`: 178 179 ```python 180 from isctest.template import NS1, NS2, zones 181 from isctest.zone import Zone, configure_root 182 183 184 def bootstrap(): 185 zone = Zone("signed-example", NS2, signed=True) 186 zone.configure() # keys, zone file, dnssec-signzone 187 188 # root zone on ns1, delegating to (and providing DS records for) 189 # the zone above 190 root = configure_root([zone]) 191 192 return { 193 "trust_anchors": root.trust_anchors(), 194 "zones": zones([root, zone]), 195 } 196 ``` 197 198 With `signed=True`, `configure()` also generates a KSK+ZSK with 199 `dnssec-keygen` and signs the rendered zone with `dnssec-signzone`. To 200 sign a hand-written zone file checked into git (no rendering at all), put 201 it at `ns2/zones/<name>.db` and drive the steps yourself: 202 203 ```python 204 def bootstrap(): 205 zone = Zone("signed-example", NS2, signed=True) 206 zone.add_keys() 207 zone.sign("-3 ABCD") # extra dnssec-signzone args, e.g. NSEC3 208 209 root = configure_root([zone]) 210 return { 211 "trust_anchors": root.trust_anchors(), 212 "zones": zones([root, zone]), 213 } 214 ``` 215 216 On the template side, the `_common` includes consume the returned data: the 217 authoritative server's `named.conf.j2` ends with 218 219 ```jinja 220 {% include "_common/controls.conf.j2" %} 221 {% include "_common/zones.conf.j2" %} 222 ``` 223 224 (`zones.conf.j2` emits a `zone` block for every zone whose nameserver 225 matches the rendering server) and the validating resolver's with 226 227 ```jinja 228 {% include "_common/controls.conf.j2" %} 229 {% include "_common/trusted.conf.j2" %} 230 {% include "_common/root.hint.conf" %} 231 ``` 232 233 (`trusted.conf.j2` emits `trust-anchors` from the `trust_anchors` data, and 234 `root.hint.conf` points the resolver at ns1 for the root zone). 235 236 The `dnssec_py` directory is the canonical example of this pattern, 237 including several modules with different zone setups sharing one directory. 238 Declare the generated files as artifacts: 239 240 ```python 241 pytestmark = pytest.mark.extra_artifacts( 242 [ 243 "ns*/dsset-*", 244 "ns*/trusted.conf", 245 "ns*/zones/*.db", 246 "ns*/zones/*.db.signed", 247 ] 248 ) 249 ``` 250 251 252 ## Drive named and watch its logs 253 254 The `NamedInstance` fixtures (`ns1` ... `ns11`) control the running servers. 255 The cardinal rule: never `time.sleep()` waiting for the server watch the 256 log for the line that proves the event happened. 257 258 Wait for a log line caused by an action enter the watcher *before* 259 triggering the action, so the line cannot slip past unobserved: 260 261 ```python 262 def test_cache_flush(ns4): 263 with ns4.watch_log_from_here() as watcher: 264 ns4.rndc("flush") 265 watcher.wait_for_line("flushing caches in all views succeeded") 266 ``` 267 268 `wait_for_line()` accepts a string, a compiled regex, or a list of either 269 (any match wins) and returns the `re.Match`. `wait_for_all([...])` waits 270 until every pattern has appeared, `wait_for_sequence([...])` requires them 271 in order. To match lines that may already have been logged (e.g. during 272 startup), use `watch_log_from_start()` instead. 273 274 Reconfigure a server mid-test by re-rendering its config template with new 275 data, then reloading. `reconfigure()` and `reload()` send the rndc command 276 and wait for the corresponding completion line in the log: 277 278 ```python 279 def test_with_new_config(ns1, templates): 280 templates.render("ns1/named.conf", {"flag": True}) 281 ns1.reconfigure() 282 ``` 283 284 Dynamic updates go through `nsupdate`: 285 286 ```python 287 import dns.update 288 289 def test_update(ns1): 290 update = dns.update.UpdateMessage("example.") 291 update.add("added.example.", 300, "A", "10.0.0.2") 292 ns1.nsupdate(update) 293 ``` 294 295 Send queries and check the responses with `isctest.query` and 296 `isctest.check`: 297 298 ```python 299 msg = isctest.query.create("a.example.", "A") 300 response = isctest.query.udp(msg, ns1.ip) # or .tcp(), .tls() 301 isctest.check.noerror(response) # rcode checks: nxdomain, 302 # servfail, refused, ... 303 isctest.check.adflag(response) # flag checks: rdflag, 304 # noraflag, ... 305 isctest.check.same_answer(response, expected) # compare two responses 306 ``` 307 308 To wait until a zone transfer has happened, poll the SOA serial: 309 310 ```python 311 isctest.query.wait_for_serial(ns2.ip, "example.", expected_serial) 312 ``` 313 314 For grepping a file the server produced (dumps, statistics), use 315 `isctest.text`: 316 317 ```python 318 from re import compile as Re 319 320 ns4.rndc("dumpdb -all") 321 dump = isctest.text.TextFile("ns4/named_dump.db") 322 assert dump.grep(Re(r"^a\.example\..*10\.0\.0\.1")) 323 ``` 324