README revision 1.11 1 1.1 christos This is a collection of tests for GDB.
2 1.1 christos
3 1.1 christos The file gdb/README contains basic instructions on how to run the
4 1.1 christos testsuite, while this file documents additional options and controls
5 1.1 christos that are available. The GDB wiki may also have some pages with ideas
6 1.1 christos and suggestions.
7 1.1 christos
8 1.1 christos
9 1.1 christos Running the Testsuite
10 1.1 christos *********************
11 1.1 christos
12 1.1 christos There are two ways to run the testsuite and pass additional parameters
13 1.1 christos to DejaGnu. The first is to do `make check' in the main build
14 1.1 christos directory and specifying the makefile variable `RUNTESTFLAGS':
15 1.1 christos
16 1.9 christos make check RUNTESTFLAGS='GDB=/usr/bin/gdb gdb.base/a2-run.exp'
17 1.1 christos
18 1.1 christos The second is to cd to the testsuite directory and invoke the DejaGnu
19 1.1 christos `runtest' command directly.
20 1.1 christos
21 1.1 christos cd testsuite
22 1.1 christos make site.exp
23 1.9 christos runtest GDB=/usr/bin/gdb
24 1.1 christos
25 1.1 christos (The `site.exp' file contains a handful of useful variables like host
26 1.1 christos and target triplets, and pathnames.)
27 1.1 christos
28 1.6 christos Parallel testing
29 1.6 christos ****************
30 1.6 christos
31 1.6 christos If not testing with a remote host (in DejaGnu's sense), you can run
32 1.6 christos the GDB test suite in a fully parallel mode. In this mode, each .exp
33 1.6 christos file runs separately and maybe simultaneously. The test suite ensures
34 1.6 christos that all the temporary files created by the test suite do not clash,
35 1.6 christos by putting them into separate directories. This mode is primarily
36 1.6 christos intended for use by the Makefile.
37 1.6 christos
38 1.6 christos For GNU make, the Makefile tries to run the tests in parallel mode if
39 1.6 christos any -j option is given. For a non-GNU make, tests are not
40 1.6 christos parallelized.
41 1.6 christos
42 1.6 christos If RUNTESTFLAGS is not empty, then by default the tests are
43 1.6 christos serialized. This can be overridden by either using the
44 1.6 christos `check-parallel' target in the Makefile, or by setting FORCE_PARALLEL
45 1.6 christos to any non-empty value:
46 1.6 christos
47 1.6 christos make check-parallel RUNTESTFLAGS="--target_board=native-gdbserver"
48 1.6 christos make check RUNTESTFLAGS="--target_board=native-gdbserver" FORCE_PARALLEL=1
49 1.6 christos
50 1.6 christos If you want to use runtest directly instead of using the Makefile, see
51 1.6 christos the description of GDB_PARALLEL below.
52 1.6 christos
53 1.6 christos Racy testcases
54 1.6 christos **************
55 1.6 christos
56 1.6 christos Sometimes, new testcases are added to the testsuite that are not
57 1.6 christos entirely deterministic, and can randomly pass or fail. We call them
58 1.6 christos "racy testcases", and they can be bothersome when one is comparing
59 1.6 christos different testsuite runs. In order to help identifying them, it is
60 1.6 christos possible to run the tests several times in a row and ask the testsuite
61 1.6 christos machinery to analyze the results. To do that, you need to specify the
62 1.6 christos RACY_ITER environment variable to make:
63 1.6 christos
64 1.6 christos make check RACY_ITER=5 -j4
65 1.6 christos
66 1.6 christos The value assigned to RACY_ITER represents the number of times you
67 1.6 christos wish to run the tests in sequence (in the example above, the entire
68 1.6 christos testsuite will be executed 5 times in a row, in parallel). It is also
69 1.6 christos possible to check just a specific test:
70 1.6 christos
71 1.6 christos make check TESTS='gdb.base/default.exp' RACY_ITER=3
72 1.6 christos
73 1.6 christos One can also decide to call the Makefile rules by hand inside the
74 1.6 christos gdb/testsuite directory, e.g.:
75 1.6 christos
76 1.10 christos make check-parallel-racy -j4
77 1.6 christos
78 1.6 christos In which case the value of the DEFAULT_RACY_ITER variable (inside
79 1.6 christos gdb/testsuite/Makefile.in) will be used to determine how many
80 1.6 christos iterations will be run.
81 1.6 christos
82 1.6 christos After running the tests, you shall see a file name 'racy.sum' in the
83 1.6 christos gdb/testsuite directory. You can also inspect the generated *.log and
84 1.6 christos *.sum files by looking into the gdb/testsuite/racy_ouputs directory.
85 1.6 christos
86 1.6 christos If you already have *.sum files generated from previous testsuite runs
87 1.6 christos and you would like to analyze them without having to run the testsuite
88 1.6 christos again, you can also use the 'analyze-racy-logs.py' script directly.
89 1.6 christos It is located in the gdb/testsuite/ directory, and it expects a list
90 1.6 christos of two or more *.sum files to be provided as its argument. For
91 1.6 christos example:
92 1.6 christos
93 1.6 christos ./gdb/testsuite/analyze-racy-logs.py testsuite-01/gdb.sum \
94 1.6 christos testsuite-02/gdb.sum testsuite-03/gdb.sum
95 1.6 christos
96 1.6 christos The script will output its analysis report to the standard output.
97 1.6 christos
98 1.9 christos Re-running Tests Outside The Testsuite
99 1.9 christos **************************************
100 1.9 christos
101 1.9 christos When running a test, the arguments used to run GDB are saved to gdb.cmd and
102 1.9 christos all commands sent to GDB are saved to gdb.in. As well as being a reference
103 1.9 christos of the commands run, they can be used to manually re-run a test by using
104 1.9 christos the gdb.in file as a batch file to a GDB launched with the arguments in the
105 1.9 christos gdb.cmd file, for example:
106 1.9 christos $(cat outputs/gdb.base/store/gdb.cmd) -x outputs/gdb.base/store/gdb.in
107 1.9 christos
108 1.9 christos Tests that run GDB multiple times will append .1, .2, .3 etc to the end
109 1.9 christos of each .cmd and .in file.
110 1.9 christos
111 1.9 christos When gdbserver is launched as part of a test, a gdbserver.cmd will be created.
112 1.9 christos To re-run these tests, run the contents of gdbserver.cmd in a separate
113 1.9 christos terminal before running gdb, for example:
114 1.9 christos $(cat outputs/gdb.base/store/gdbserver.cmd)
115 1.9 christos Alternatively, if the test is run with GDBSERVER_DEBUG="replay", then this
116 1.9 christos will create a gdbserver.replay file which can be used with the gdbreplay tool,
117 1.9 christos instead of launching gdbserver.
118 1.9 christos
119 1.1 christos Running the Performance Tests
120 1.1 christos *****************************
121 1.1 christos
122 1.1 christos GDB Testsuite includes performance test cases, which are not run together
123 1.1 christos with other test cases, because performance test cases are slow and need
124 1.1 christos a quiet system. There are two ways to run the performance test cases.
125 1.1 christos The first is to do `make check-perf' in the main build directory:
126 1.1 christos
127 1.1 christos make check-perf RUNTESTFLAGS="solib.exp SOLIB_COUNT=8"
128 1.1 christos
129 1.1 christos The second is to cd to the testsuite directory and invoke the DejaGnu
130 1.1 christos `runtest' command directly.
131 1.1 christos
132 1.1 christos cd testsuite
133 1.1 christos make site.exp
134 1.1 christos runtest GDB_PERFTEST_MODE=both GDB_PERFTEST_TIMEOUT=4000 --directory=gdb.perf solib.exp SOLIB_COUNT=8
135 1.1 christos
136 1.1 christos Only "compile", "run" and "both" are valid to GDB_PERFTEST_MODE. They
137 1.1 christos stand for "compile tests only", "run tests only", and "compile and run
138 1.1 christos tests" respectively. "both" is the default. GDB_PERFTEST_TIMEOUT
139 1.1 christos specify the timeout, which is 3000 in default. The result of
140 1.1 christos performance test is appended in `testsuite/perftest.log'.
141 1.1 christos
142 1.1 christos Testsuite Parameters
143 1.1 christos ********************
144 1.1 christos
145 1.1 christos The following parameters are DejaGNU variables that you can set to
146 1.1 christos affect the testsuite run globally.
147 1.1 christos
148 1.1 christos GDB
149 1.1 christos
150 1.1 christos By default, the testsuite exercises the GDB in the build directory,
151 1.1 christos but you can set GDB to be a pathname to a different version. For
152 1.1 christos instance,
153 1.1 christos
154 1.1 christos make check RUNTESTFLAGS=GDB=/usr/bin/gdb
155 1.1 christos
156 1.1 christos runs the testsuite on the GDB in /usr/bin.
157 1.1 christos
158 1.1 christos GDBSERVER
159 1.1 christos
160 1.1 christos You can set GDBSERVER to be a particular GDBserver of interest, so for
161 1.1 christos instance
162 1.1 christos
163 1.1 christos make check RUNTESTFLAGS="GDB=/usr/bin/gdb GDBSERVER=/usr/bin/gdbserver"
164 1.1 christos
165 1.1 christos checks both the installed GDB and GDBserver.
166 1.1 christos
167 1.10 christos GDB_DATA_DIRECTORY
168 1.10 christos
169 1.10 christos If you set GDB, then by default the testsuite assumes you are
170 1.10 christos exercising an installed GDB, and thus the testsuite lets GDB use its
171 1.10 christos configured data directory. Otherwise, if you don't set GDB, then by
172 1.10 christos default the tested GDB uses the data directory found under the GDB
173 1.10 christos build directory. You can override this by setting GDB_DATA_DIRECTORY.
174 1.10 christos For instance:
175 1.10 christos
176 1.10 christos make check \
177 1.10 christos RUNTESTFLAGS="GDB=/path/to/other/build/gdb \
178 1.10 christos GDB_DATA_DIRECTORY=/path/to/other/build/gdb/data-directory"
179 1.10 christos
180 1.1 christos INTERNAL_GDBFLAGS
181 1.1 christos
182 1.1 christos Command line options passed to all GDB invocations.
183 1.10 christos The default is set in lib/gdb.exp.
184 1.1 christos
185 1.1 christos This is actually considered an internal variable, and you
186 1.1 christos won't normally want to change it. However, in some situations,
187 1.1 christos this may be tweaked as a last resort if the testsuite doesn't
188 1.1 christos have direct support for the specifics of your environment.
189 1.1 christos The testsuite does not override a value provided by the user.
190 1.1 christos
191 1.1 christos As an example, when testing an installed GDB that has been
192 1.1 christos configured with `--with-system-gdbinit', like by default,
193 1.1 christos you do not want ~/.gdbinit to interfere with tests, but, you
194 1.1 christos may want the system .gdbinit file loaded. As there's no way to
195 1.1 christos ask the testsuite, or GDB, to load the system gdbinit but
196 1.1 christos not ~/.gdbinit, a workaround is then to remove `-nx' from
197 1.1 christos INTERNAL_GDBFLAGS, and point $HOME at a directory without
198 1.1 christos a .gdbinit. For example:
199 1.1 christos
200 1.1 christos cd testsuite
201 1.1 christos HOME=`pwd` runtest \
202 1.1 christos GDB=/usr/bin/gdb \
203 1.1 christos GDBSERVER=/usr/bin/gdbserver \
204 1.11 christos INTERNAL_GDBFLAGS="-nw -q -iex 'set height 0' -iex 'set width 0'"
205 1.10 christos
206 1.10 christos Note that we do not need to specify '-data-directory' here
207 1.10 christos as we are testing an installed GDB.
208 1.1 christos
209 1.1 christos GDB_PARALLEL
210 1.1 christos
211 1.9 christos To use parallel testing mode without using the Makefile, set
212 1.6 christos GDB_PARALLEL on the runtest command line to "yes". Before starting
213 1.6 christos the tests, you must ensure that the directories cache, outputs, and
214 1.6 christos temp in the test suite build directory are either empty or have been
215 1.6 christos deleted. cache in particular is used to share data across invocations
216 1.6 christos of runtest, and files there may affect the test results. The Makefile
217 1.6 christos automatically does these deletions.
218 1.6 christos
219 1.6 christos FORCE_PARALLEL
220 1.6 christos
221 1.6 christos Setting FORCE_PARALLEL to any non-empty value forces parallel testing
222 1.6 christos mode even if RUNTESTFLAGS is not empty.
223 1.6 christos
224 1.6 christos FORCE_SEPARATE_MI_TTY
225 1.6 christos
226 1.6 christos Setting FORCE_MI_SEPARATE_UI to 1 forces all MI testing to start GDB
227 1.6 christos in console mode, with MI running on a separate TTY, on a secondary UI
228 1.6 christos started with "new-ui".
229 1.1 christos
230 1.1 christos GDB_INOTIFY
231 1.1 christos
232 1.1 christos For debugging parallel mode, it is handy to be able to see when a test
233 1.1 christos case writes to a file outside of its designated output directory.
234 1.1 christos
235 1.1 christos If you have the inotify-tools package installed, you can set the
236 1.1 christos GDB_INOTIFY variable on the runtest command line. This will cause the
237 1.1 christos test suite to watch for parallel-unsafe file creations and report
238 1.1 christos them, both to stdout and in the test suite log file.
239 1.1 christos
240 1.1 christos This setting is only meaningful in conjunction with GDB_PARALLEL.
241 1.1 christos
242 1.3 christos TESTS
243 1.3 christos
244 1.3 christos This variable is used to specify which set of tests to run.
245 1.3 christos It is passed to make (not runtest) and its contents are a space separated
246 1.3 christos list of tests to run.
247 1.3 christos
248 1.3 christos If using GNU make then the contents are wildcard-expanded using
249 1.3 christos GNU make's $(wildcard) function. Test paths must be fully specified,
250 1.3 christos relative to the "testsuite" subdirectory. This allows one to run all
251 1.3 christos tests in a subdirectory by passing "gdb.subdir/*.exp", or more simply
252 1.3 christos by using the check-gdb.subdir target in the Makefile.
253 1.3 christos
254 1.3 christos If for some strange reason one wanted to run all tests that begin with
255 1.3 christos the letter "d" that is also possible: TESTS="*/d*.exp".
256 1.3 christos
257 1.3 christos Do not write */*.exp to specify all tests (assuming all tests are only
258 1.3 christos nested one level deep, which is not necessarily true). This will pick up
259 1.3 christos .exp files in ancillary directories like "lib" and "config".
260 1.3 christos Instead write gdb.*/*.exp.
261 1.3 christos
262 1.3 christos Example:
263 1.3 christos
264 1.3 christos make -j10 check TESTS="gdb.server/[s-w]*.exp */x*.exp"
265 1.3 christos
266 1.3 christos If not using GNU make then the value is passed directly to runtest.
267 1.3 christos If not specified, all tests are run.
268 1.3 christos
269 1.3 christos READ1
270 1.3 christos
271 1.3 christos This make (not runtest) variable is used to specify whether the
272 1.3 christos testsuite preloads the read1.so library into expect. Any non-empty
273 1.3 christos value means true. See "Race detection" below.
274 1.3 christos
275 1.8 christos GDB_TEST_SOCKETHOST
276 1.8 christos
277 1.8 christos This variable can provide the hostname/address that should be used
278 1.8 christos when performing GDBserver-related tests. This is useful in some
279 1.8 christos situations, e.g., when you want to test the IPv6 connectivity of GDB
280 1.8 christos and GDBserver, or when using a different hostname/address is needed.
281 1.8 christos For example, to make GDB and GDBserver use IPv6-only connections, you
282 1.8 christos can do:
283 1.8 christos
284 1.8 christos make check TESTS="gdb.server/*.exp" RUNTESTFLAGS='GDB_TEST_SOCKETHOST=tcp6:[::1]'
285 1.8 christos
286 1.8 christos Note that only a hostname/address can be provided, without a port
287 1.8 christos number.
288 1.8 christos
289 1.8 christos TS
290 1.8 christos
291 1.8 christos This variable turns on the timestamp printing for each line of "make
292 1.8 christos check". Note that the timestamp will be printed on stdout output
293 1.8 christos only. In other words, there will be no timestamp output on either
294 1.8 christos gdb.sum and gdb.log files. If you would like to enable timestamp
295 1.8 christos printing, you can do:
296 1.8 christos
297 1.8 christos make check TS=1
298 1.8 christos
299 1.8 christos TS_FORMAT
300 1.8 christos
301 1.8 christos You can provide a custom format for timestamp printing with this
302 1.8 christos variable. The format must be a string compatible with "strftime".
303 1.8 christos This variable is only useful when the TS variable is also provided.
304 1.8 christos If you would like to change the output format of the timestamp, you
305 1.8 christos can do:
306 1.8 christos
307 1.8 christos make check TS=1 TS_FORMAT='[%b %H:%S]'
308 1.8 christos
309 1.9 christos GDB_DEBUG
310 1.9 christos
311 1.9 christos When set gdb debug is sent to the file gdb.debug in the test output
312 1.9 christos directory. It should be set to a comma separated list of gdb debug
313 1.9 christos components.
314 1.9 christos For example, to turn on debugging for infrun and target, you can do:
315 1.9 christos
316 1.9 christos make check GDB_DEBUG="infrun,target"
317 1.9 christos
318 1.9 christos GDBSERVER_DEBUG
319 1.9 christos
320 1.9 christos When set gdbserver debug is sent to the a file in the test output directory.
321 1.9 christos It should be set to a comma separated list of the following options:
322 1.9 christos debug - write gdbserver debug to gdbserver.debug.
323 1.9 christos remote - write gdbserver remote debug to gdbserver.debug.
324 1.9 christos replay - write a replay log to the file gdbserver.replay for use
325 1.9 christos with gdbreplay.
326 1.9 christos Alternatively, it can be set to "all" to turn on all the above
327 1.9 christos For example, to turn on gdbserver debugging, you can do:
328 1.9 christos
329 1.9 christos make check GDBSERVER_DEBUG="debug,replay"
330 1.9 christos
331 1.11 christos GDB_TARGET_USERNAME
332 1.11 christos GDB_HOST_USERNAME
333 1.11 christos
334 1.11 christos These settings are only used with the check-all-boards target, and
335 1.11 christos should be the usernames of two separate users on the local machine,
336 1.11 christos both of which the current user can ssh to without a password.
337 1.11 christos
338 1.11 christos These users will be used by board files that simulate remote targets
339 1.11 christos by switching to a different user on the same machine. These users
340 1.11 christos will have random files copied into their $HOME directories, so it is a
341 1.11 christos good idea to setup new users just for this purpose.
342 1.11 christos
343 1.11 christos Testing All Simple Boards
344 1.11 christos *************************
345 1.11 christos
346 1.11 christos There are a number of boards that ship with GDB that simulate common
347 1.11 christos debug scenarios. For example by sshing to a different user on the
348 1.11 christos local machine and running gdbserver as this alternative user we aim to
349 1.11 christos simulate a true remote debug experience.
350 1.11 christos
351 1.11 christos There is a script binutils-gdb/gdb/testssuite/make-check-all.sh which
352 1.11 christos can be used to run a defined set of tests using all of the available
353 1.11 christos simple board files. Support for using this script is also included in
354 1.11 christos GDB's makefile, and can be used as:
355 1.11 christos
356 1.11 christos make check-all-boards GDB_TARGET_USERNAME=remote-target \
357 1.11 christos GDB_HOST_USERNAME=remote-host \
358 1.11 christos TESTS="gdb.base/break.exp"
359 1.11 christos
360 1.11 christos The 'remote-target' and 'remote-host' can be replaced with any user
361 1.11 christos names on the local machine, the only requirements are that the current
362 1.11 christos user must be able to ssh to these users without a password, and these
363 1.11 christos users must be happy to have arbitrary files copied into their $HOME
364 1.11 christos directory. Ideally, these users would be setup just for GDB testing.
365 1.11 christos
366 1.11 christos The check-all-boards target requires that TESTS be defined, though it
367 1.11 christos is fine to include multiple tests.
368 1.11 christos
369 1.11 christos The results are preserved, and can be found in the directory
370 1.11 christos gdb/testsuite/check-all/. The results are split by the board file
371 1.11 christos used.
372 1.11 christos
373 1.10 christos Architecture-specific Parameters
374 1.10 christos ******************************
375 1.10 christos
376 1.10 christos This section documents architecture-specific parameters that can be used with
377 1.10 christos the GDB testsuite.
378 1.10 christos
379 1.10 christos - AArch64 (Linux)
380 1.10 christos
381 1.10 christos ARM_CC_FOR_TARGET
382 1.10 christos
383 1.10 christos The AArch64 ports of GDB and GDBserver support debugging AArch32
384 1.10 christos 32-bit programs running on 64-bit state. There are some tests under
385 1.10 christos gdb.multi/ that exercise this particular feature.
386 1.10 christos
387 1.10 christos By default, the testsuite tries to find a compiler capable of
388 1.10 christos generating 32-bit executables. If no compiler is found, or if the
389 1.10 christos 32-bit executable generated by the found compiler can't be executed
390 1.10 christos correctly, the tests will be marked UNSUPPORTED. The list of 32-bit
391 1.10 christos Arm compiler names the testsuite will try can be found in
392 1.10 christos gdb/testsuite/lib/gdb.exp:arm_cc_for_target.
393 1.10 christos
394 1.10 christos You can set ARM_CC_FOR_TARGET to override the search and explicitly
395 1.10 christos specify the compiler to use. This variable should contain the command
396 1.10 christos line for the compiler, including the full path to it, if the compiler
397 1.10 christos is not in $PATH.
398 1.10 christos
399 1.10 christos Example:
400 1.10 christos
401 1.10 christos make check-gdb TESTS="gdb.multi/multi-arch.exp" RUNTESTFLAGS="ARM_CC_FOR_TARGET=arm-linux-gnueabihf-gcc"
402 1.10 christos
403 1.3 christos Race detection
404 1.3 christos **************
405 1.3 christos
406 1.3 christos The testsuite includes a mechanism that helps detect test races.
407 1.3 christos
408 1.3 christos For example, say the program running under expect outputs "abcd", and
409 1.3 christos a test does something like this:
410 1.3 christos
411 1.3 christos expect {
412 1.3 christos "a.*c" {
413 1.3 christos }
414 1.3 christos "b" {
415 1.3 christos }
416 1.3 christos "a" {
417 1.3 christos }
418 1.3 christos }
419 1.3 christos
420 1.3 christos Which case happens to match depends on what expect manages to read
421 1.3 christos into its internal buffer in one go. If it manages to read three bytes
422 1.3 christos or more, then the first case matches. If it manages to read two
423 1.3 christos bytes, then the second case matches. If it manages to read only one
424 1.3 christos byte, then the third case matches.
425 1.3 christos
426 1.3 christos To help detect these cases, the race detection mechanism preloads a
427 1.3 christos library into expect that forces the `read' system call to always
428 1.3 christos return at most 1 byte.
429 1.3 christos
430 1.3 christos To enable this, either pass a non-empty value in the READ1 make
431 1.3 christos variable, or use the check-read1 make target instead of check.
432 1.3 christos
433 1.10 christos Example:
434 1.3 christos
435 1.3 christos make -j10 check-read1 TESTS="*/paginate-*.exp"
436 1.10 christos
437 1.10 christos If you've already built the read1 support code, either via a previous
438 1.10 christos 'check-read1' run, or by using "make read1", you can use:
439 1.10 christos
440 1.3 christos make -j10 check READ1="1"
441 1.1 christos
442 1.9 christos Note: While the intention is to detect races and make otherwise passing tests
443 1.9 christos fail, it can also have the effect of making otherwise failing tests pass.
444 1.9 christos This happens f.i. if the test is trying to match a gdb prompt using an end of
445 1.9 christos input marker "${gdb_prompt} $" and there is output after the gdb prompt. This
446 1.9 christos may either pass or fail in normal operation, but using check-read1 will ensure
447 1.10 christos that it passes. Use check-readmore to detect this type of failure.
448 1.9 christos
449 1.1 christos Testsuite Configuration
450 1.1 christos ***********************
451 1.1 christos
452 1.1 christos It is possible to adjust the behavior of the testsuite by defining
453 1.1 christos the global variables listed below, either in a `site.exp' file,
454 1.1 christos or in a board file.
455 1.1 christos
456 1.1 christos gdb_test_timeout
457 1.1 christos
458 1.1 christos Defining this variable changes the default timeout duration used
459 1.1 christos during communication with GDB. More specifically, the global variable
460 1.1 christos used during testing is `timeout', but this variable gets reset to
461 1.1 christos `gdb_test_timeout' at the beginning of each testcase, which ensures
462 1.1 christos that any local change to `timeout' in a testcase does not affect
463 1.1 christos subsequent testcases.
464 1.1 christos
465 1.1 christos This global variable comes in handy when the debugger is slower than
466 1.1 christos normal due to the testing environment, triggering unexpected `TIMEOUT'
467 1.1 christos test failures. Examples include when testing on a remote machine, or
468 1.1 christos against a system where communications are slow.
469 1.1 christos
470 1.1 christos If not specifically defined, this variable gets automatically defined
471 1.1 christos to the same value as `timeout' during the testsuite initialization.
472 1.1 christos The default value of the timeout is defined in the file
473 1.1 christos `testsuite/config/unix.exp' (at least for Unix hosts; board files may
474 1.1 christos have their own values).
475 1.1 christos
476 1.3 christos gdb_reverse_timeout
477 1.3 christos
478 1.3 christos Defining this variable changes the default timeout duration when tests
479 1.3 christos under gdb.reverse directory are running. Process record and reverse
480 1.3 christos debugging is so slow that its tests have unexpected `TIMEOUT' test
481 1.3 christos failures. This global variable is useful to bump up the value of
482 1.3 christos `timeout' for gdb.reverse tests and doesn't cause any delay where
483 1.3 christos actual failures happen in the rest of the testsuite.
484 1.3 christos
485 1.1 christos
486 1.1 christos Board Settings
487 1.1 christos **************
488 1.1 christos
489 1.1 christos DejaGNU includes the concept of a "board file", which specifies
490 1.1 christos testing details for a particular target (which are often bare circuit
491 1.1 christos boards, thus the name).
492 1.1 christos
493 1.1 christos In the GDB testsuite specifically, the board file may include a
494 1.1 christos number of "board settings" that test cases may check before deciding
495 1.1 christos whether to exercise a particular feature. For instance, a board
496 1.1 christos lacking any I/O devices, or perhaps simply having its I/O devices
497 1.1 christos not wired up, should set `noinferiorio'.
498 1.1 christos
499 1.1 christos Here are the supported board settings:
500 1.1 christos
501 1.1 christos gdb,cannot_call_functions
502 1.1 christos
503 1.1 christos The board does not support inferior call, that is, invoking inferior
504 1.1 christos functions in GDB.
505 1.1 christos
506 1.1 christos gdb,can_reverse
507 1.1 christos
508 1.1 christos The board supports reverse execution.
509 1.1 christos
510 1.1 christos gdb,no_hardware_watchpoints
511 1.1 christos
512 1.1 christos The board does not support hardware watchpoints.
513 1.1 christos
514 1.1 christos gdb,nofileio
515 1.1 christos
516 1.1 christos GDB is unable to intercept target file operations in remote and
517 1.1 christos perform them on the host.
518 1.1 christos
519 1.1 christos gdb,noinferiorio
520 1.1 christos
521 1.1 christos The board is unable to provide I/O capability to the inferior.
522 1.1 christos
523 1.1 christos gdb,noresults
524 1.1 christos
525 1.1 christos A program will not return an exit code or result code (or the value
526 1.1 christos of the result is undefined, and should not be looked at).
527 1.1 christos
528 1.1 christos gdb,nosignals
529 1.1 christos
530 1.1 christos The board does not support signals.
531 1.1 christos
532 1.1 christos gdb,skip_huge_test
533 1.1 christos
534 1.1 christos Skip time-consuming tests on the board with slow connection.
535 1.1 christos
536 1.1 christos gdb,skip_float_tests
537 1.1 christos
538 1.1 christos Skip tests related to floating point.
539 1.1 christos
540 1.1 christos gdb,use_precord
541 1.1 christos
542 1.1 christos The board supports process record.
543 1.1 christos
544 1.3 christos gdb_init_command
545 1.3 christos gdb_init_commands
546 1.3 christos
547 1.3 christos Commands to send to GDB every time a program is about to be run. The
548 1.3 christos first of these settings defines a single command as a string. The
549 1.3 christos second defines a TCL list of commands being a string each. The commands
550 1.3 christos are sent one by one in a sequence, first from `gdb_init_command', if any,
551 1.3 christos followed by individual commands from `gdb_init_command', if any, in this
552 1.3 christos list's order.
553 1.3 christos
554 1.1 christos gdb_server_prog
555 1.1 christos
556 1.1 christos The location of GDBserver. If GDBserver somewhere other than its
557 1.1 christos default location is used in test, specify the location of GDBserver in
558 1.1 christos this variable. The location is a file name for GDBserver, and may be
559 1.1 christos either absolute or relative to the testsuite subdirectory of the build
560 1.1 christos directory.
561 1.1 christos
562 1.1 christos in_proc_agent
563 1.1 christos
564 1.1 christos The location of the in-process agent (used for fast tracepoints and
565 1.1 christos other special tests). If the in-process agent of interest is anywhere
566 1.1 christos other than its default location, set this variable. The location is a
567 1.1 christos filename, and may be either absolute or relative to the testsuite
568 1.1 christos subdirectory of the build directory.
569 1.1 christos
570 1.1 christos noargs
571 1.1 christos
572 1.1 christos GDB does not support argument passing for inferior.
573 1.1 christos
574 1.1 christos no_long_long
575 1.1 christos
576 1.1 christos The board does not support type long long.
577 1.1 christos
578 1.1 christos use_cygmon
579 1.1 christos
580 1.1 christos The board is running the monitor Cygmon.
581 1.1 christos
582 1.1 christos use_gdb_stub
583 1.1 christos
584 1.1 christos The tests are running with a GDB stub.
585 1.1 christos
586 1.1 christos exit_is_reliable
587 1.1 christos
588 1.1 christos Set to true if GDB can assume that letting the program run to end
589 1.1 christos reliably results in program exits being reported as such, as opposed
590 1.1 christos to, e.g., the program ending in an infinite loop or the board
591 1.1 christos crashing/resetting. If not set, this defaults to $use_gdb_stub. In
592 1.1 christos other words, native targets are assumed reliable by default, and
593 1.1 christos remote stubs assumed unreliable.
594 1.1 christos
595 1.1 christos gdb,predefined_tsv
596 1.1 christos
597 1.1 christos The predefined trace state variables the board has.
598 1.1 christos
599 1.6 christos gdb,no_thread_names
600 1.6 christos
601 1.6 christos The target doesn't support thread names.
602 1.1 christos
603 1.9 christos gdb,pie_flag
604 1.9 christos
605 1.9 christos The flag required to force the compiler to produce position-independent
606 1.9 christos executables.
607 1.9 christos
608 1.9 christos gdb,pie_ldflag
609 1.9 christos
610 1.9 christos The flag required to force the linker to produce position-independent
611 1.9 christos executables.
612 1.9 christos
613 1.8 christos gdb,nopie_flag
614 1.8 christos
615 1.8 christos The flag required to force the compiler to produce non-position-independent
616 1.8 christos executables.
617 1.8 christos
618 1.10 christos gdb,nopie_ldflag
619 1.10 christos
620 1.10 christos The flag required to force the linker to produce non-position-independent
621 1.10 christos executables.
622 1.10 christos
623 1.9 christos gdb,debug
624 1.9 christos
625 1.9 christos When set gdb debug is sent to the file gdb.debug in the test output
626 1.9 christos directory. It should be set to a comma separated list of gdb debug
627 1.9 christos components. For example, to turn on debugging for infrun and target, set to
628 1.9 christos "infrun,target".
629 1.9 christos
630 1.9 christos gdbserver,debug
631 1.9 christos
632 1.9 christos When set gdbserver debug is sent to the file gdbserver.debug in the test
633 1.9 christos output directory. For valid values see the entry for GDBSERVER_DEBUG.
634 1.9 christos
635 1.1 christos Testsuite Organization
636 1.1 christos **********************
637 1.1 christos
638 1.1 christos The testsuite is entirely contained in `gdb/testsuite'. The main
639 1.1 christos directory of the testsuite includes some makefiles and configury, but
640 1.1 christos these are minimal, and used for little besides cleaning up, since the
641 1.1 christos tests themselves handle the compilation of the programs that GDB will
642 1.1 christos run.
643 1.1 christos
644 1.1 christos The file `testsuite/lib/gdb.exp' contains common utility procs useful
645 1.1 christos for all GDB tests, while the directory testsuite/config contains
646 1.1 christos configuration-specific files, typically used for special-purpose
647 1.1 christos definitions of procs like `gdb_load' and `gdb_start'.
648 1.1 christos
649 1.1 christos The tests themselves are to be found in directories named
650 1.1 christos 'testsuite/gdb.* and subdirectories of those. The names of the test
651 1.1 christos files must always end with ".exp". DejaGNU collects the test files by
652 1.1 christos wildcarding in the test directories, so both subdirectories and
653 1.1 christos individual files typically get chosen and run in alphabetical order.
654 1.1 christos
655 1.1 christos The following lists some notable types of subdirectories and what they
656 1.1 christos are for. Since DejaGNU finds test files no matter where they are
657 1.1 christos located, and since each test file sets up its own compilation and
658 1.1 christos execution environment, this organization is simply for convenience and
659 1.1 christos intelligibility.
660 1.1 christos
661 1.1 christos gdb.base
662 1.1 christos
663 1.1 christos This is the base testsuite. The tests in it should apply to all
664 1.1 christos configurations of GDB (but generic native-only tests may live here).
665 1.1 christos The test programs should be in the subset of C that is both valid
666 1.1 christos ANSI/ISO C, and C++.
667 1.1 christos
668 1.1 christos gdb.<lang>
669 1.1 christos
670 1.1 christos Language-specific tests for any language besides C. Examples are
671 1.7 christos gdb.cp for C++ and gdb.rust for Rust.
672 1.1 christos
673 1.1 christos gdb.<platform>
674 1.1 christos
675 1.1 christos Non-portable tests. The tests are specific to a specific
676 1.5 christos configuration (host or target), such as eCos.
677 1.1 christos
678 1.1 christos gdb.arch
679 1.1 christos
680 1.1 christos Architecture-specific tests that are (usually) cross-platform.
681 1.1 christos
682 1.1 christos gdb.<subsystem>
683 1.1 christos
684 1.1 christos Tests that exercise a specific GDB subsystem in more depth. For
685 1.1 christos instance, gdb.disasm exercises various disassemblers, while
686 1.1 christos gdb.stabs tests pathways through the stabs symbol reader.
687 1.1 christos
688 1.1 christos gdb.perf
689 1.1 christos
690 1.1 christos GDB performance tests.
691 1.1 christos
692 1.1 christos Writing Tests
693 1.1 christos *************
694 1.1 christos
695 1.1 christos In many areas, the GDB tests are already quite comprehensive; you
696 1.1 christos should be able to copy existing tests to handle new cases. Be aware
697 1.1 christos that older tests may use obsolete practices but have not yet been
698 1.1 christos updated.
699 1.1 christos
700 1.1 christos You should try to use `gdb_test' whenever possible, since it includes
701 1.1 christos cases to handle all the unexpected errors that might happen. However,
702 1.1 christos it doesn't cost anything to add new test procedures; for instance,
703 1.1 christos gdb.base/exprs.exp defines a `test_expr' that calls `gdb_test'
704 1.1 christos multiple times.
705 1.1 christos
706 1.1 christos Only use `send_gdb' and `gdb_expect' when absolutely necessary. Even
707 1.1 christos if GDB has several valid responses to a command, you can use
708 1.1 christos `gdb_test_multiple'. Like `gdb_test', `gdb_test_multiple' recognizes
709 1.1 christos internal errors and unexpected prompts.
710 1.1 christos
711 1.1 christos Do not write tests which expect a literal tab character from GDB. On
712 1.1 christos some operating systems (e.g. OpenBSD) the TTY layer expands tabs to
713 1.1 christos spaces, so by the time GDB's output reaches `expect' the tab is gone.
714 1.1 christos
715 1.1 christos The source language programs do *not* need to be in a consistent
716 1.1 christos style. Since GDB is used to debug programs written in many different
717 1.1 christos styles, it's worth having a mix of styles in the testsuite; for
718 1.1 christos instance, some GDB bugs involving the display of source lines might
719 1.1 christos never manifest themselves if the test programs used GNU coding style
720 1.1 christos uniformly.
721 1.1 christos
722 1.1 christos Some testcase results need more detailed explanation:
723 1.1 christos
724 1.1 christos KFAIL
725 1.1 christos
726 1.1 christos Use KFAIL for known problem of GDB itself. You must specify the GDB
727 1.1 christos bug report number, as in these sample tests:
728 1.1 christos
729 1.1 christos kfail "gdb/13392" "continue to marker 2"
730 1.1 christos
731 1.1 christos or
732 1.1 christos
733 1.1 christos setup_kfail gdb/13392 "*-*-*"
734 1.1 christos kfail "continue to marker 2"
735 1.1 christos
736 1.1 christos
737 1.1 christos XFAIL
738 1.1 christos
739 1.1 christos Short for "expected failure", this indicates a known problem with the
740 1.1 christos environment. This could include limitations of the operating system,
741 1.1 christos compiler version, and other components.
742 1.1 christos
743 1.1 christos This example from gdb.base/attach-pie-misread.exp is a sanity check
744 1.1 christos for the target environment:
745 1.1 christos
746 1.1 christos # On x86_64 it is commonly about 4MB.
747 1.1 christos if {$stub_size > 25000000} {
748 1.1 christos xfail "stub size $stub_size is too large"
749 1.1 christos return
750 1.1 christos }
751 1.1 christos
752 1.1 christos You should provide bug report number for the failing component of the
753 1.1 christos environment, if such bug report is available, as with this example
754 1.1 christos referring to a GCC problem:
755 1.1 christos
756 1.1 christos if {[test_compiler_info {gcc-[0-3]-*}]
757 1.1 christos || [test_compiler_info {gcc-4-[0-5]-*}]} {
758 1.1 christos setup_xfail "gcc/46955" *-*-*
759 1.1 christos }
760 1.1 christos gdb_test "python print ttype.template_argument(2)" "&C::c"
761 1.1 christos
762 1.1 christos Note that it is also acceptable, and often preferable, to avoid
763 1.1 christos running the test at all. This is the better option if the limitation
764 1.1 christos is intrinsic to the environment, rather than a bug expected to be
765 1.1 christos fixed in the near future.
766 1.8 christos
767 1.8 christos Local vs Remote vs Native
768 1.8 christos *************************
769 1.8 christos
770 1.8 christos It's unfortunately easy to get confused in the testsuite about what's
771 1.8 christos native and what's not, what's remote and what's not. The confusion is
772 1.8 christos caused by the overlap in vocabulary between DejaGnu and GDB.
773 1.8 christos
774 1.8 christos From a DejaGnu point of view:
775 1.8 christos
776 1.8 christos - native: the host or target board is considered native if the its
777 1.8 christos triplet is the same as the build system's triplet,
778 1.8 christos
779 1.8 christos - remote: the host or target board is considered remote if it's
780 1.8 christos running on a different machine, and thus require ssh, for example,
781 1.8 christos to run commands, versus simply running commands directly.
782 1.8 christos
783 1.8 christos Note that they are not mutually exclusive, as you can have a remote
784 1.8 christos machine that has the same triplet as the build machine.
785 1.8 christos
786 1.8 christos From a GDB point of view:
787 1.8 christos
788 1.8 christos - native: when GDB uses system calls such as ptrace to interact
789 1.8 christos directly with processes on the same system its running on,
790 1.8 christos
791 1.8 christos - remote: when GDB speaks the RSP (Remote Serial Protocol) with
792 1.8 christos another program doing the ptrace stuff.
793 1.8 christos
794 1.8 christos Note that they are mutually exclusive. An inferior can only be either
795 1.8 christos debugged with the native target, or with the remote target a specific
796 1.8 christos time.
797 1.8 christos
798 1.8 christos That means that there are cases where the target is not remote for
799 1.8 christos DejaGnu, but is remote for GDB (e.g. running GDBserver on the same
800 1.8 christos machine).
801 1.8 christos
802 1.8 christos You can also have a remote target for DejaGnu, but native for GDB
803 1.8 christos (e.g. building on x86 a GDB that runs on ARM and running the
804 1.8 christos testsuite with a remote host).
805 1.8 christos
806 1.8 christos Therefore, care must be taken to check for the right kind of remote.
807 1.8 christos Use [is_remote target] to check whether the DejaGnu target board is
808 1.8 christos remote. When what you really want to know is whether GDB is using the
809 1.8 christos remote protocol, because feature X is only available when GDB debugs
810 1.8 christos natively, check gdb_protocol instead.
811