COOKBOOK.md revision 1.1 1 1.1 christos <!--
2 1.1 christos Copyright (C) Internet Systems Consortium, Inc. ("ISC")
3 1.1 christos
4 1.1 christos SPDX-License-Identifier: MPL-2.0
5 1.1 christos
6 1.1 christos This Source Code Form is subject to the terms of the Mozilla Public
7 1.1 christos License, v. 2.0. If a copy of the MPL was not distributed with this
8 1.1 christos file, you can obtain one at https://mozilla.org/MPL/2.0/.
9 1.1 christos
10 1.1 christos See the COPYRIGHT file distributed with this work for additional
11 1.1 christos information regarding copyright ownership.
12 1.1 christos -->
13 1.1 christos
14 1.1 christos # BIND9 System Test Cookbook
15 1.1 christos
16 1.1 christos Task-oriented recipes for the system test framework. Each recipe is
17 1.1 christos self-contained and shows working code that can be copied and adapted.
18 1.1 christos For concepts and reference documentation (build setup, fixtures, runner
19 1.1 christos internals), see [README.md](README.md).
20 1.1 christos
21 1.1 christos Prerequisite for all recipes: a built tree and the test network interfaces
22 1.1 christos configured see "Running the Tests" in the README.
23 1.1 christos
24 1.1 christos
25 1.1 christos ## Iterate on a single test
26 1.1 christos
27 1.1 christos The basic loop while developing a test:
28 1.1 christos
29 1.1 christos ```sh
30 1.1 christos cd bin/tests/system
31 1.1 christos pytest mytest # whole directory
32 1.1 christos pytest mytest/tests_mytestmod.py::test_one_case # one test function
33 1.1 christos ```
34 1.1 christos
35 1.1 christos When a test fails, its temporary directory is kept and a stable symlink to it
36 1.1 christos is created (e.g. `mytest-mytestmod` for `mytest/tests_mytestmod.py`). Look
37 1.1 christos there for `pytest.log.txt` (the test's own log) and `ns*/named.run` (each
38 1.1 christos server's debug-level log). Pass `--noclean` to keep the directory even on
39 1.1 christos success.
40 1.1 christos
41 1.1 christos To poke at the servers interactively, pause the test at any point by
42 1.1 christos inserting a breakpoint, or start pdb right away with `--trace`; to get a pdb
43 1.1 christos prompt automatically when a test fails, use `--pdb`. In all cases the
44 1.1 christos servers keep running while the test is paused, so their state can be
45 1.1 christos inspected:
46 1.1 christos
47 1.1 christos ```sh
48 1.1 christos pytest mytest --trace # pdb at the start of each test
49 1.1 christos pytest mytest --pdb # pdb when a test fails
50 1.1 christos ```
51 1.1 christos
52 1.1 christos Timing-sensitive bugs hide in system tests: before declaring a test done, run
53 1.1 christos it several times (and ideally once under parallel load, `pytest -n auto`).
54 1.1 christos A test that flakes is treated as a bug in the test.
55 1.1 christos
56 1.1 christos
57 1.1 christos ## Add a new system test directory
58 1.1 christos
59 1.1 christos This skeleton sets up one authoritative server and queries it. Pick a name
60 1.1 christos that starts with a letter and uses underscores as word separators (hyphens
61 1.1 christos are not allowed), and create:
62 1.1 christos
63 1.1 christos ```
64 1.1 christos demo/
65 1.1 christos ns1/
66 1.1 christos named.conf.j2
67 1.1 christos example.db
68 1.1 christos tests_demo.py
69 1.1 christos ```
70 1.1 christos
71 1.1 christos `demo/ns1/named.conf.j2` the config template; the runner renders it to
72 1.1 christos `named.conf` at setup time, filling in the assigned ports. Templates inside
73 1.1 christos an `nsN`/`ansN` subdirectory also get an `ns` variable describing that
74 1.1 christos server, so the config doesn't hardcode its own address (`@ns.ip@` renders to
75 1.1 christos 10.53.0.1 in ns1, 10.53.0.2 in ns2, ...):
76 1.1 christos
77 1.1 christos ```jinja
78 1.1 christos options {
79 1.1 christos query-source address @ns.ip@;
80 1.1 christos notify-source @ns.ip@;
81 1.1 christos transfer-source @ns.ip@;
82 1.1 christos port @PORT@;
83 1.1 christos pid-file "named.pid";
84 1.1 christos listen-on { @ns.ip@; };
85 1.1 christos listen-on-v6 { none; };
86 1.1 christos recursion no;
87 1.1 christos dnssec-validation no;
88 1.1 christos };
89 1.1 christos
90 1.1 christos {% include "_common/controls.conf.j2" %}
91 1.1 christos
92 1.1 christos zone "example" {
93 1.1 christos type primary;
94 1.1 christos file "example.db";
95 1.1 christos };
96 1.1 christos ```
97 1.1 christos
98 1.1 christos The `_common/controls.conf.j2` include sets up the rndc control channel, so
99 1.1 christos the test (and the runner's shutdown sequence) can use `rndc`.
100 1.1 christos
101 1.1 christos `demo/ns1/example.db` a plain zone file:
102 1.1 christos
103 1.1 christos ```
104 1.1 christos $TTL 300
105 1.1 christos example. IN SOA ns1.example. hostmaster.example. 1 600 600 1200 600
106 1.1 christos example. NS ns1.example.
107 1.1 christos ns1.example. A 10.53.0.1
108 1.1 christos a.example. A 10.0.0.1
109 1.1 christos ```
110 1.1 christos
111 1.1 christos `demo/tests_demo.py` the test module:
112 1.1 christos
113 1.1 christos ```python
114 1.1 christos import isctest
115 1.1 christos
116 1.1 christos
117 1.1 christos def test_a_record(ns1):
118 1.1 christos msg = isctest.query.create("a.example.", "A")
119 1.1 christos response = isctest.query.udp(msg, ns1.ip)
120 1.1 christos isctest.check.noerror(response)
121 1.1 christos assert str(response.answer[0][0]) == "10.0.0.1"
122 1.1 christos ```
123 1.1 christos
124 1.1 christos That's it run it with `pytest demo`. The pytest runner discovers new
125 1.1 christos directories automatically; to also include the test in `make check` runs,
126 1.1 christos add it to `TESTS` in `Makefile.am`.
127 1.1 christos
128 1.1 christos Notes:
129 1.1 christos
130 1.1 christos - The `ns1` fixture is the started server (an
131 1.1 christos `isctest.instance.NamedInstance`); `ns1.ip` is 10.53.0.1. Servers in
132 1.1 christos `nsN`/`ansN` subdirectories are started automatically.
133 1.1 christos - If your test writes files beyond the usual ones (e.g. `dig.out.*`,
134 1.1 christos `dsset-*`), declare them as a module-level marker
135 1.1 christos `pytestmark = pytest.mark.extra_artifacts(["dig.out.*"])` or the
136 1.1 christos artifact check at teardown will fail. Most real test modules carry one.
137 1.1 christos
138 1.1 christos
139 1.1 christos ## Set up zones in bootstrap()
140 1.1 christos
141 1.1 christos A module-level `bootstrap()` function runs before the config templates are
142 1.1 christos rendered and before the servers start. This is where zone files and DNSSEC
143 1.1 christos keys can be generated; whatever dict it returns becomes template data.
144 1.1 christos
145 1.1 christos The simplest case is a plain unsigned zone use this instead of writing
146 1.1 christos the same boilerplate zone file by hand in every `nsN` directory:
147 1.1 christos
148 1.1 christos ```python
149 1.1 christos from isctest.template import NS1, zones
150 1.1 christos from isctest.zone import Zone
151 1.1 christos
152 1.1 christos
153 1.1 christos def bootstrap():
154 1.1 christos zone = Zone("example", NS1) # signed=False is the default
155 1.1 christos zone.configure() # render the zone file
156 1.1 christos return {"zones": zones([zone])}
157 1.1 christos ```
158 1.1 christos
159 1.1 christos `Zone.configure()` renders the zone file from
160 1.1 christos `_common/zones/template.db.j2.manual`, a generic zone with SOA, NS, and a
161 1.1 christos few test records. To provide your own content, drop a template named after
162 1.1 christos the zone file `ns1/zones/<name>.db.j2.manual` and `configure()` picks
163 1.1 christos it up automatically instead of the generic one. (The `.j2.manual` suffix
164 1.1 christos keeps the file out of the runner's automatic template pass, which renders
165 1.1 christos every plain `*.j2` at setup time; here `bootstrap()` renders it instead.)
166 1.1 christos
167 1.1 christos For DNSSEC-signed zones, pass `signed=True` and make sure to configure the
168 1.1 christos `trust_anchors`:
169 1.1 christos
170 1.1 christos ```python
171 1.1 christos from isctest.template import NS1, NS2, zones
172 1.1 christos from isctest.zone import Zone, configure_root
173 1.1 christos
174 1.1 christos
175 1.1 christos def bootstrap():
176 1.1 christos zone = Zone("signed-example", NS2, signed=True)
177 1.1 christos zone.configure() # keys, zone file, dnssec-signzone
178 1.1 christos
179 1.1 christos # root zone on ns1, delegating to (and providing DS records for)
180 1.1 christos # the zone above
181 1.1 christos root = configure_root([zone])
182 1.1 christos
183 1.1 christos return {
184 1.1 christos "trust_anchors": root.trust_anchors(),
185 1.1 christos "zones": zones([root, zone]),
186 1.1 christos }
187 1.1 christos ```
188 1.1 christos
189 1.1 christos With `signed=True`, `configure()` also generates a KSK+ZSK with
190 1.1 christos `dnssec-keygen` and signs the rendered zone with `dnssec-signzone`. To
191 1.1 christos sign a hand-written zone file checked into git (no rendering at all), put
192 1.1 christos it at `ns2/zones/<name>.db` and drive the steps yourself:
193 1.1 christos
194 1.1 christos ```python
195 1.1 christos def bootstrap():
196 1.1 christos zone = Zone("signed-example", NS2, signed=True)
197 1.1 christos zone.add_keys()
198 1.1 christos zone.sign("-3 ABCD") # extra dnssec-signzone args, e.g. NSEC3
199 1.1 christos
200 1.1 christos root = configure_root([zone])
201 1.1 christos return {
202 1.1 christos "trust_anchors": root.trust_anchors(),
203 1.1 christos "zones": zones([root, zone]),
204 1.1 christos }
205 1.1 christos ```
206 1.1 christos
207 1.1 christos On the template side, the `_common` includes consume the returned data: the
208 1.1 christos authoritative server's `named.conf.j2` ends with
209 1.1 christos
210 1.1 christos ```jinja
211 1.1 christos {% include "_common/controls.conf.j2" %}
212 1.1 christos {% include "_common/zones.conf.j2" %}
213 1.1 christos ```
214 1.1 christos
215 1.1 christos (`zones.conf.j2` emits a `zone` block for every zone whose nameserver
216 1.1 christos matches the rendering server) and the validating resolver's with
217 1.1 christos
218 1.1 christos ```jinja
219 1.1 christos {% include "_common/controls.conf.j2" %}
220 1.1 christos {% include "_common/trusted.conf.j2" %}
221 1.1 christos {% include "_common/root.hint.conf" %}
222 1.1 christos ```
223 1.1 christos
224 1.1 christos (`trusted.conf.j2` emits `trust-anchors` from the `trust_anchors` data, and
225 1.1 christos `root.hint.conf` points the resolver at ns1 for the root zone).
226 1.1 christos
227 1.1 christos The `dnssec_py` directory is the canonical example of this pattern,
228 1.1 christos including several modules with different zone setups sharing one directory.
229 1.1 christos Declare the generated files as artifacts:
230 1.1 christos
231 1.1 christos ```python
232 1.1 christos pytestmark = pytest.mark.extra_artifacts(
233 1.1 christos [
234 1.1 christos "ns*/dsset-*",
235 1.1 christos "ns*/trusted.conf",
236 1.1 christos "ns*/zones/*.db",
237 1.1 christos "ns*/zones/*.db.signed",
238 1.1 christos ]
239 1.1 christos )
240 1.1 christos ```
241 1.1 christos
242 1.1 christos
243 1.1 christos ## Drive named and watch its logs
244 1.1 christos
245 1.1 christos The `NamedInstance` fixtures (`ns1` ... `ns11`) control the running servers.
246 1.1 christos The cardinal rule: never `time.sleep()` waiting for the server watch the
247 1.1 christos log for the line that proves the event happened.
248 1.1 christos
249 1.1 christos Wait for a log line caused by an action enter the watcher *before*
250 1.1 christos triggering the action, so the line cannot slip past unobserved:
251 1.1 christos
252 1.1 christos ```python
253 1.1 christos def test_cache_flush(ns4):
254 1.1 christos with ns4.watch_log_from_here() as watcher:
255 1.1 christos ns4.rndc("flush")
256 1.1 christos watcher.wait_for_line("flushing caches in all views succeeded")
257 1.1 christos ```
258 1.1 christos
259 1.1 christos `wait_for_line()` accepts a string, a compiled regex, or a list of either
260 1.1 christos (any match wins) and returns the `re.Match`. `wait_for_all([...])` waits
261 1.1 christos until every pattern has appeared, `wait_for_sequence([...])` requires them
262 1.1 christos in order. To match lines that may already have been logged (e.g. during
263 1.1 christos startup), use `watch_log_from_start()` instead.
264 1.1 christos
265 1.1 christos Reconfigure a server mid-test by re-rendering its config template with new
266 1.1 christos data, then reloading. `reconfigure()` and `reload()` send the rndc command
267 1.1 christos and wait for the corresponding completion line in the log:
268 1.1 christos
269 1.1 christos ```python
270 1.1 christos def test_with_new_config(ns1, templates):
271 1.1 christos templates.render("ns1/named.conf", {"flag": True})
272 1.1 christos ns1.reconfigure()
273 1.1 christos ```
274 1.1 christos
275 1.1 christos Dynamic updates go through `nsupdate`:
276 1.1 christos
277 1.1 christos ```python
278 1.1 christos import dns.update
279 1.1 christos
280 1.1 christos def test_update(ns1):
281 1.1 christos update = dns.update.UpdateMessage("example.")
282 1.1 christos update.add("added.example.", 300, "A", "10.0.0.2")
283 1.1 christos ns1.nsupdate(update)
284 1.1 christos ```
285 1.1 christos
286 1.1 christos Send queries and check the responses with `isctest.query` and
287 1.1 christos `isctest.check`:
288 1.1 christos
289 1.1 christos ```python
290 1.1 christos msg = isctest.query.create("a.example.", "A")
291 1.1 christos response = isctest.query.udp(msg, ns1.ip) # or .tcp(), .tls()
292 1.1 christos isctest.check.noerror(response) # rcode checks: nxdomain,
293 1.1 christos # servfail, refused, ...
294 1.1 christos isctest.check.adflag(response) # flag checks: rdflag,
295 1.1 christos # noraflag, ...
296 1.1 christos isctest.check.same_answer(response, expected) # compare two responses
297 1.1 christos ```
298 1.1 christos
299 1.1 christos To wait until a zone transfer has happened, poll the SOA serial:
300 1.1 christos
301 1.1 christos ```python
302 1.1 christos isctest.query.wait_for_serial(ns2.ip, "example.", expected_serial)
303 1.1 christos ```
304 1.1 christos
305 1.1 christos For grepping a file the server produced (dumps, statistics), use
306 1.1 christos `isctest.text`:
307 1.1 christos
308 1.1 christos ```python
309 1.1 christos from re import compile as Re
310 1.1 christos
311 1.1 christos ns4.rndc("dumpdb -all")
312 1.1 christos dump = isctest.text.TextFile("ns4/named_dump.db")
313 1.1 christos assert dump.grep(Re(r"^a\.example\..*10\.0\.0\.1"))
314 1.1 christos ```
315