1 1.1 christos This is a collection of tests for GDB. 2 1.1 christos 3 1.1 christos The file gdb/README contains basic instructions on how to run the 4 1.1 christos testsuite, while this file documents additional options and controls 5 1.1 christos that are available. The GDB wiki may also have some pages with ideas 6 1.1 christos and suggestions. 7 1.1 christos 8 1.1 christos 9 1.1 christos Running the Testsuite 10 1.1 christos ********************* 11 1.1 christos 12 1.1 christos There are two ways to run the testsuite and pass additional parameters 13 1.1 christos to DejaGnu. The first is to do `make check' in the main build 14 1.1 christos directory and specifying the makefile variable `RUNTESTFLAGS': 15 1.1 christos 16 1.9 christos make check RUNTESTFLAGS='GDB=/usr/bin/gdb gdb.base/a2-run.exp' 17 1.1 christos 18 1.1 christos The second is to cd to the testsuite directory and invoke the DejaGnu 19 1.1 christos `runtest' command directly. 20 1.1 christos 21 1.1 christos cd testsuite 22 1.1 christos make site.exp 23 1.9 christos runtest GDB=/usr/bin/gdb 24 1.1 christos 25 1.1 christos (The `site.exp' file contains a handful of useful variables like host 26 1.1 christos and target triplets, and pathnames.) 27 1.1 christos 28 1.6 christos Parallel testing 29 1.6 christos **************** 30 1.6 christos 31 1.6 christos If not testing with a remote host (in DejaGnu's sense), you can run 32 1.6 christos the GDB test suite in a fully parallel mode. In this mode, each .exp 33 1.6 christos file runs separately and maybe simultaneously. The test suite ensures 34 1.6 christos that all the temporary files created by the test suite do not clash, 35 1.6 christos by putting them into separate directories. This mode is primarily 36 1.6 christos intended for use by the Makefile. 37 1.6 christos 38 1.6 christos For GNU make, the Makefile tries to run the tests in parallel mode if 39 1.6 christos any -j option is given. For a non-GNU make, tests are not 40 1.6 christos parallelized. 41 1.6 christos 42 1.6 christos If RUNTESTFLAGS is not empty, then by default the tests are 43 1.6 christos serialized. This can be overridden by either using the 44 1.6 christos `check-parallel' target in the Makefile, or by setting FORCE_PARALLEL 45 1.6 christos to any non-empty value: 46 1.6 christos 47 1.6 christos make check-parallel RUNTESTFLAGS="--target_board=native-gdbserver" 48 1.6 christos make check RUNTESTFLAGS="--target_board=native-gdbserver" FORCE_PARALLEL=1 49 1.6 christos 50 1.6 christos If you want to use runtest directly instead of using the Makefile, see 51 1.6 christos the description of GDB_PARALLEL below. 52 1.6 christos 53 1.6 christos Racy testcases 54 1.6 christos ************** 55 1.6 christos 56 1.6 christos Sometimes, new testcases are added to the testsuite that are not 57 1.6 christos entirely deterministic, and can randomly pass or fail. We call them 58 1.6 christos "racy testcases", and they can be bothersome when one is comparing 59 1.6 christos different testsuite runs. In order to help identifying them, it is 60 1.6 christos possible to run the tests several times in a row and ask the testsuite 61 1.6 christos machinery to analyze the results. To do that, you need to specify the 62 1.6 christos RACY_ITER environment variable to make: 63 1.6 christos 64 1.6 christos make check RACY_ITER=5 -j4 65 1.6 christos 66 1.6 christos The value assigned to RACY_ITER represents the number of times you 67 1.6 christos wish to run the tests in sequence (in the example above, the entire 68 1.6 christos testsuite will be executed 5 times in a row, in parallel). It is also 69 1.6 christos possible to check just a specific test: 70 1.6 christos 71 1.6 christos make check TESTS='gdb.base/default.exp' RACY_ITER=3 72 1.6 christos 73 1.6 christos One can also decide to call the Makefile rules by hand inside the 74 1.6 christos gdb/testsuite directory, e.g.: 75 1.6 christos 76 1.10 christos make check-parallel-racy -j4 77 1.6 christos 78 1.6 christos In which case the value of the DEFAULT_RACY_ITER variable (inside 79 1.6 christos gdb/testsuite/Makefile.in) will be used to determine how many 80 1.6 christos iterations will be run. 81 1.6 christos 82 1.6 christos After running the tests, you shall see a file name 'racy.sum' in the 83 1.6 christos gdb/testsuite directory. You can also inspect the generated *.log and 84 1.6 christos *.sum files by looking into the gdb/testsuite/racy_ouputs directory. 85 1.6 christos 86 1.6 christos If you already have *.sum files generated from previous testsuite runs 87 1.6 christos and you would like to analyze them without having to run the testsuite 88 1.6 christos again, you can also use the 'analyze-racy-logs.py' script directly. 89 1.6 christos It is located in the gdb/testsuite/ directory, and it expects a list 90 1.6 christos of two or more *.sum files to be provided as its argument. For 91 1.6 christos example: 92 1.6 christos 93 1.6 christos ./gdb/testsuite/analyze-racy-logs.py testsuite-01/gdb.sum \ 94 1.6 christos testsuite-02/gdb.sum testsuite-03/gdb.sum 95 1.6 christos 96 1.6 christos The script will output its analysis report to the standard output. 97 1.6 christos 98 1.9 christos Re-running Tests Outside The Testsuite 99 1.9 christos ************************************** 100 1.9 christos 101 1.9 christos When running a test, the arguments used to run GDB are saved to gdb.cmd and 102 1.9 christos all commands sent to GDB are saved to gdb.in. As well as being a reference 103 1.9 christos of the commands run, they can be used to manually re-run a test by using 104 1.9 christos the gdb.in file as a batch file to a GDB launched with the arguments in the 105 1.9 christos gdb.cmd file, for example: 106 1.9 christos $(cat outputs/gdb.base/store/gdb.cmd) -x outputs/gdb.base/store/gdb.in 107 1.9 christos 108 1.9 christos Tests that run GDB multiple times will append .1, .2, .3 etc to the end 109 1.9 christos of each .cmd and .in file. 110 1.9 christos 111 1.9 christos When gdbserver is launched as part of a test, a gdbserver.cmd will be created. 112 1.9 christos To re-run these tests, run the contents of gdbserver.cmd in a separate 113 1.9 christos terminal before running gdb, for example: 114 1.9 christos $(cat outputs/gdb.base/store/gdbserver.cmd) 115 1.9 christos Alternatively, if the test is run with GDBSERVER_DEBUG="replay", then this 116 1.9 christos will create a gdbserver.replay file which can be used with the gdbreplay tool, 117 1.9 christos instead of launching gdbserver. 118 1.9 christos 119 1.1 christos Running the Performance Tests 120 1.1 christos ***************************** 121 1.1 christos 122 1.1 christos GDB Testsuite includes performance test cases, which are not run together 123 1.1 christos with other test cases, because performance test cases are slow and need 124 1.1 christos a quiet system. There are two ways to run the performance test cases. 125 1.1 christos The first is to do `make check-perf' in the main build directory: 126 1.1 christos 127 1.1 christos make check-perf RUNTESTFLAGS="solib.exp SOLIB_COUNT=8" 128 1.1 christos 129 1.1 christos The second is to cd to the testsuite directory and invoke the DejaGnu 130 1.1 christos `runtest' command directly. 131 1.1 christos 132 1.1 christos cd testsuite 133 1.1 christos make site.exp 134 1.1 christos runtest GDB_PERFTEST_MODE=both GDB_PERFTEST_TIMEOUT=4000 --directory=gdb.perf solib.exp SOLIB_COUNT=8 135 1.1 christos 136 1.1 christos Only "compile", "run" and "both" are valid to GDB_PERFTEST_MODE. They 137 1.1 christos stand for "compile tests only", "run tests only", and "compile and run 138 1.1 christos tests" respectively. "both" is the default. GDB_PERFTEST_TIMEOUT 139 1.1 christos specify the timeout, which is 3000 in default. The result of 140 1.1 christos performance test is appended in `testsuite/perftest.log'. 141 1.1 christos 142 1.1 christos Testsuite Parameters 143 1.1 christos ******************** 144 1.1 christos 145 1.1 christos The following parameters are DejaGNU variables that you can set to 146 1.1 christos affect the testsuite run globally. 147 1.1 christos 148 1.1 christos GDB 149 1.1 christos 150 1.1 christos By default, the testsuite exercises the GDB in the build directory, 151 1.1 christos but you can set GDB to be a pathname to a different version. For 152 1.1 christos instance, 153 1.1 christos 154 1.1 christos make check RUNTESTFLAGS=GDB=/usr/bin/gdb 155 1.1 christos 156 1.1 christos runs the testsuite on the GDB in /usr/bin. 157 1.1 christos 158 1.1 christos GDBSERVER 159 1.1 christos 160 1.1 christos You can set GDBSERVER to be a particular GDBserver of interest, so for 161 1.1 christos instance 162 1.1 christos 163 1.1 christos make check RUNTESTFLAGS="GDB=/usr/bin/gdb GDBSERVER=/usr/bin/gdbserver" 164 1.1 christos 165 1.1 christos checks both the installed GDB and GDBserver. 166 1.1 christos 167 1.10 christos GDB_DATA_DIRECTORY 168 1.10 christos 169 1.10 christos If you set GDB, then by default the testsuite assumes you are 170 1.10 christos exercising an installed GDB, and thus the testsuite lets GDB use its 171 1.10 christos configured data directory. Otherwise, if you don't set GDB, then by 172 1.10 christos default the tested GDB uses the data directory found under the GDB 173 1.10 christos build directory. You can override this by setting GDB_DATA_DIRECTORY. 174 1.10 christos For instance: 175 1.10 christos 176 1.10 christos make check \ 177 1.10 christos RUNTESTFLAGS="GDB=/path/to/other/build/gdb \ 178 1.10 christos GDB_DATA_DIRECTORY=/path/to/other/build/gdb/data-directory" 179 1.10 christos 180 1.1 christos INTERNAL_GDBFLAGS 181 1.1 christos 182 1.1 christos Command line options passed to all GDB invocations. 183 1.10 christos The default is set in lib/gdb.exp. 184 1.1 christos 185 1.1 christos This is actually considered an internal variable, and you 186 1.1 christos won't normally want to change it. However, in some situations, 187 1.1 christos this may be tweaked as a last resort if the testsuite doesn't 188 1.1 christos have direct support for the specifics of your environment. 189 1.1 christos The testsuite does not override a value provided by the user. 190 1.1 christos 191 1.1 christos As an example, when testing an installed GDB that has been 192 1.1 christos configured with `--with-system-gdbinit', like by default, 193 1.1 christos you do not want ~/.gdbinit to interfere with tests, but, you 194 1.1 christos may want the system .gdbinit file loaded. As there's no way to 195 1.1 christos ask the testsuite, or GDB, to load the system gdbinit but 196 1.1 christos not ~/.gdbinit, a workaround is then to remove `-nx' from 197 1.1 christos INTERNAL_GDBFLAGS, and point $HOME at a directory without 198 1.1 christos a .gdbinit. For example: 199 1.1 christos 200 1.1 christos cd testsuite 201 1.1 christos HOME=`pwd` runtest \ 202 1.1 christos GDB=/usr/bin/gdb \ 203 1.1 christos GDBSERVER=/usr/bin/gdbserver \ 204 1.11 christos INTERNAL_GDBFLAGS="-nw -q -iex 'set height 0' -iex 'set width 0'" 205 1.10 christos 206 1.10 christos Note that we do not need to specify '-data-directory' here 207 1.10 christos as we are testing an installed GDB. 208 1.1 christos 209 1.1 christos GDB_PARALLEL 210 1.1 christos 211 1.9 christos To use parallel testing mode without using the Makefile, set 212 1.6 christos GDB_PARALLEL on the runtest command line to "yes". Before starting 213 1.6 christos the tests, you must ensure that the directories cache, outputs, and 214 1.6 christos temp in the test suite build directory are either empty or have been 215 1.6 christos deleted. cache in particular is used to share data across invocations 216 1.6 christos of runtest, and files there may affect the test results. The Makefile 217 1.6 christos automatically does these deletions. 218 1.6 christos 219 1.6 christos FORCE_PARALLEL 220 1.6 christos 221 1.6 christos Setting FORCE_PARALLEL to any non-empty value forces parallel testing 222 1.6 christos mode even if RUNTESTFLAGS is not empty. 223 1.6 christos 224 1.6 christos FORCE_SEPARATE_MI_TTY 225 1.6 christos 226 1.6 christos Setting FORCE_MI_SEPARATE_UI to 1 forces all MI testing to start GDB 227 1.6 christos in console mode, with MI running on a separate TTY, on a secondary UI 228 1.6 christos started with "new-ui". 229 1.1 christos 230 1.1 christos GDB_INOTIFY 231 1.1 christos 232 1.1 christos For debugging parallel mode, it is handy to be able to see when a test 233 1.1 christos case writes to a file outside of its designated output directory. 234 1.1 christos 235 1.1 christos If you have the inotify-tools package installed, you can set the 236 1.1 christos GDB_INOTIFY variable on the runtest command line. This will cause the 237 1.1 christos test suite to watch for parallel-unsafe file creations and report 238 1.1 christos them, both to stdout and in the test suite log file. 239 1.1 christos 240 1.1 christos This setting is only meaningful in conjunction with GDB_PARALLEL. 241 1.1 christos 242 1.3 christos TESTS 243 1.3 christos 244 1.3 christos This variable is used to specify which set of tests to run. 245 1.3 christos It is passed to make (not runtest) and its contents are a space separated 246 1.3 christos list of tests to run. 247 1.3 christos 248 1.3 christos If using GNU make then the contents are wildcard-expanded using 249 1.3 christos GNU make's $(wildcard) function. Test paths must be fully specified, 250 1.3 christos relative to the "testsuite" subdirectory. This allows one to run all 251 1.3 christos tests in a subdirectory by passing "gdb.subdir/*.exp", or more simply 252 1.3 christos by using the check-gdb.subdir target in the Makefile. 253 1.3 christos 254 1.3 christos If for some strange reason one wanted to run all tests that begin with 255 1.3 christos the letter "d" that is also possible: TESTS="*/d*.exp". 256 1.3 christos 257 1.3 christos Do not write */*.exp to specify all tests (assuming all tests are only 258 1.3 christos nested one level deep, which is not necessarily true). This will pick up 259 1.3 christos .exp files in ancillary directories like "lib" and "config". 260 1.3 christos Instead write gdb.*/*.exp. 261 1.3 christos 262 1.3 christos Example: 263 1.3 christos 264 1.3 christos make -j10 check TESTS="gdb.server/[s-w]*.exp */x*.exp" 265 1.3 christos 266 1.3 christos If not using GNU make then the value is passed directly to runtest. 267 1.3 christos If not specified, all tests are run. 268 1.3 christos 269 1.3 christos READ1 270 1.3 christos 271 1.3 christos This make (not runtest) variable is used to specify whether the 272 1.3 christos testsuite preloads the read1.so library into expect. Any non-empty 273 1.3 christos value means true. See "Race detection" below. 274 1.3 christos 275 1.8 christos GDB_TEST_SOCKETHOST 276 1.8 christos 277 1.8 christos This variable can provide the hostname/address that should be used 278 1.8 christos when performing GDBserver-related tests. This is useful in some 279 1.8 christos situations, e.g., when you want to test the IPv6 connectivity of GDB 280 1.8 christos and GDBserver, or when using a different hostname/address is needed. 281 1.8 christos For example, to make GDB and GDBserver use IPv6-only connections, you 282 1.8 christos can do: 283 1.8 christos 284 1.8 christos make check TESTS="gdb.server/*.exp" RUNTESTFLAGS='GDB_TEST_SOCKETHOST=tcp6:[::1]' 285 1.8 christos 286 1.8 christos Note that only a hostname/address can be provided, without a port 287 1.8 christos number. 288 1.8 christos 289 1.8 christos TS 290 1.8 christos 291 1.8 christos This variable turns on the timestamp printing for each line of "make 292 1.8 christos check". Note that the timestamp will be printed on stdout output 293 1.8 christos only. In other words, there will be no timestamp output on either 294 1.8 christos gdb.sum and gdb.log files. If you would like to enable timestamp 295 1.8 christos printing, you can do: 296 1.8 christos 297 1.8 christos make check TS=1 298 1.8 christos 299 1.8 christos TS_FORMAT 300 1.8 christos 301 1.8 christos You can provide a custom format for timestamp printing with this 302 1.8 christos variable. The format must be a string compatible with "strftime". 303 1.8 christos This variable is only useful when the TS variable is also provided. 304 1.8 christos If you would like to change the output format of the timestamp, you 305 1.8 christos can do: 306 1.8 christos 307 1.8 christos make check TS=1 TS_FORMAT='[%b %H:%S]' 308 1.8 christos 309 1.9 christos GDB_DEBUG 310 1.9 christos 311 1.9 christos When set gdb debug is sent to the file gdb.debug in the test output 312 1.9 christos directory. It should be set to a comma separated list of gdb debug 313 1.9 christos components. 314 1.9 christos For example, to turn on debugging for infrun and target, you can do: 315 1.9 christos 316 1.9 christos make check GDB_DEBUG="infrun,target" 317 1.9 christos 318 1.9 christos GDBSERVER_DEBUG 319 1.9 christos 320 1.9 christos When set gdbserver debug is sent to the a file in the test output directory. 321 1.9 christos It should be set to a comma separated list of the following options: 322 1.9 christos debug - write gdbserver debug to gdbserver.debug. 323 1.9 christos remote - write gdbserver remote debug to gdbserver.debug. 324 1.9 christos replay - write a replay log to the file gdbserver.replay for use 325 1.9 christos with gdbreplay. 326 1.9 christos Alternatively, it can be set to "all" to turn on all the above 327 1.9 christos For example, to turn on gdbserver debugging, you can do: 328 1.9 christos 329 1.9 christos make check GDBSERVER_DEBUG="debug,replay" 330 1.9 christos 331 1.11 christos GDB_TARGET_USERNAME 332 1.11 christos GDB_HOST_USERNAME 333 1.11 christos 334 1.11 christos These settings are only used with the check-all-boards target, and 335 1.11 christos should be the usernames of two separate users on the local machine, 336 1.11 christos both of which the current user can ssh to without a password. 337 1.11 christos 338 1.11 christos These users will be used by board files that simulate remote targets 339 1.11 christos by switching to a different user on the same machine. These users 340 1.11 christos will have random files copied into their $HOME directories, so it is a 341 1.11 christos good idea to setup new users just for this purpose. 342 1.11 christos 343 1.11 christos Testing All Simple Boards 344 1.11 christos ************************* 345 1.11 christos 346 1.11 christos There are a number of boards that ship with GDB that simulate common 347 1.11 christos debug scenarios. For example by sshing to a different user on the 348 1.11 christos local machine and running gdbserver as this alternative user we aim to 349 1.11 christos simulate a true remote debug experience. 350 1.11 christos 351 1.11 christos There is a script binutils-gdb/gdb/testssuite/make-check-all.sh which 352 1.11 christos can be used to run a defined set of tests using all of the available 353 1.11 christos simple board files. Support for using this script is also included in 354 1.11 christos GDB's makefile, and can be used as: 355 1.11 christos 356 1.11 christos make check-all-boards GDB_TARGET_USERNAME=remote-target \ 357 1.11 christos GDB_HOST_USERNAME=remote-host \ 358 1.11 christos TESTS="gdb.base/break.exp" 359 1.11 christos 360 1.11 christos The 'remote-target' and 'remote-host' can be replaced with any user 361 1.11 christos names on the local machine, the only requirements are that the current 362 1.11 christos user must be able to ssh to these users without a password, and these 363 1.11 christos users must be happy to have arbitrary files copied into their $HOME 364 1.11 christos directory. Ideally, these users would be setup just for GDB testing. 365 1.11 christos 366 1.11 christos The check-all-boards target requires that TESTS be defined, though it 367 1.11 christos is fine to include multiple tests. 368 1.11 christos 369 1.11 christos The results are preserved, and can be found in the directory 370 1.11 christos gdb/testsuite/check-all/. The results are split by the board file 371 1.11 christos used. 372 1.11 christos 373 1.10 christos Architecture-specific Parameters 374 1.10 christos ****************************** 375 1.10 christos 376 1.10 christos This section documents architecture-specific parameters that can be used with 377 1.10 christos the GDB testsuite. 378 1.10 christos 379 1.10 christos - AArch64 (Linux) 380 1.10 christos 381 1.10 christos ARM_CC_FOR_TARGET 382 1.10 christos 383 1.10 christos The AArch64 ports of GDB and GDBserver support debugging AArch32 384 1.10 christos 32-bit programs running on 64-bit state. There are some tests under 385 1.10 christos gdb.multi/ that exercise this particular feature. 386 1.10 christos 387 1.10 christos By default, the testsuite tries to find a compiler capable of 388 1.10 christos generating 32-bit executables. If no compiler is found, or if the 389 1.10 christos 32-bit executable generated by the found compiler can't be executed 390 1.10 christos correctly, the tests will be marked UNSUPPORTED. The list of 32-bit 391 1.10 christos Arm compiler names the testsuite will try can be found in 392 1.10 christos gdb/testsuite/lib/gdb.exp:arm_cc_for_target. 393 1.10 christos 394 1.10 christos You can set ARM_CC_FOR_TARGET to override the search and explicitly 395 1.10 christos specify the compiler to use. This variable should contain the command 396 1.10 christos line for the compiler, including the full path to it, if the compiler 397 1.10 christos is not in $PATH. 398 1.10 christos 399 1.10 christos Example: 400 1.10 christos 401 1.10 christos make check-gdb TESTS="gdb.multi/multi-arch.exp" RUNTESTFLAGS="ARM_CC_FOR_TARGET=arm-linux-gnueabihf-gcc" 402 1.10 christos 403 1.3 christos Race detection 404 1.3 christos ************** 405 1.3 christos 406 1.3 christos The testsuite includes a mechanism that helps detect test races. 407 1.3 christos 408 1.3 christos For example, say the program running under expect outputs "abcd", and 409 1.3 christos a test does something like this: 410 1.3 christos 411 1.3 christos expect { 412 1.3 christos "a.*c" { 413 1.3 christos } 414 1.3 christos "b" { 415 1.3 christos } 416 1.3 christos "a" { 417 1.3 christos } 418 1.3 christos } 419 1.3 christos 420 1.3 christos Which case happens to match depends on what expect manages to read 421 1.3 christos into its internal buffer in one go. If it manages to read three bytes 422 1.3 christos or more, then the first case matches. If it manages to read two 423 1.3 christos bytes, then the second case matches. If it manages to read only one 424 1.3 christos byte, then the third case matches. 425 1.3 christos 426 1.3 christos To help detect these cases, the race detection mechanism preloads a 427 1.3 christos library into expect that forces the `read' system call to always 428 1.3 christos return at most 1 byte. 429 1.3 christos 430 1.3 christos To enable this, either pass a non-empty value in the READ1 make 431 1.3 christos variable, or use the check-read1 make target instead of check. 432 1.3 christos 433 1.10 christos Example: 434 1.3 christos 435 1.3 christos make -j10 check-read1 TESTS="*/paginate-*.exp" 436 1.10 christos 437 1.10 christos If you've already built the read1 support code, either via a previous 438 1.10 christos 'check-read1' run, or by using "make read1", you can use: 439 1.10 christos 440 1.3 christos make -j10 check READ1="1" 441 1.1 christos 442 1.9 christos Note: While the intention is to detect races and make otherwise passing tests 443 1.9 christos fail, it can also have the effect of making otherwise failing tests pass. 444 1.9 christos This happens f.i. if the test is trying to match a gdb prompt using an end of 445 1.9 christos input marker "${gdb_prompt} $" and there is output after the gdb prompt. This 446 1.9 christos may either pass or fail in normal operation, but using check-read1 will ensure 447 1.10 christos that it passes. Use check-readmore to detect this type of failure. 448 1.9 christos 449 1.1 christos Testsuite Configuration 450 1.1 christos *********************** 451 1.1 christos 452 1.1 christos It is possible to adjust the behavior of the testsuite by defining 453 1.1 christos the global variables listed below, either in a `site.exp' file, 454 1.1 christos or in a board file. 455 1.1 christos 456 1.1 christos gdb_test_timeout 457 1.1 christos 458 1.1 christos Defining this variable changes the default timeout duration used 459 1.1 christos during communication with GDB. More specifically, the global variable 460 1.1 christos used during testing is `timeout', but this variable gets reset to 461 1.1 christos `gdb_test_timeout' at the beginning of each testcase, which ensures 462 1.1 christos that any local change to `timeout' in a testcase does not affect 463 1.1 christos subsequent testcases. 464 1.1 christos 465 1.1 christos This global variable comes in handy when the debugger is slower than 466 1.1 christos normal due to the testing environment, triggering unexpected `TIMEOUT' 467 1.1 christos test failures. Examples include when testing on a remote machine, or 468 1.1 christos against a system where communications are slow. 469 1.1 christos 470 1.1 christos If not specifically defined, this variable gets automatically defined 471 1.1 christos to the same value as `timeout' during the testsuite initialization. 472 1.1 christos The default value of the timeout is defined in the file 473 1.1 christos `testsuite/config/unix.exp' (at least for Unix hosts; board files may 474 1.1 christos have their own values). 475 1.1 christos 476 1.3 christos gdb_reverse_timeout 477 1.3 christos 478 1.3 christos Defining this variable changes the default timeout duration when tests 479 1.3 christos under gdb.reverse directory are running. Process record and reverse 480 1.3 christos debugging is so slow that its tests have unexpected `TIMEOUT' test 481 1.3 christos failures. This global variable is useful to bump up the value of 482 1.3 christos `timeout' for gdb.reverse tests and doesn't cause any delay where 483 1.3 christos actual failures happen in the rest of the testsuite. 484 1.3 christos 485 1.1 christos 486 1.1 christos Board Settings 487 1.1 christos ************** 488 1.1 christos 489 1.1 christos DejaGNU includes the concept of a "board file", which specifies 490 1.1 christos testing details for a particular target (which are often bare circuit 491 1.1 christos boards, thus the name). 492 1.1 christos 493 1.1 christos In the GDB testsuite specifically, the board file may include a 494 1.1 christos number of "board settings" that test cases may check before deciding 495 1.1 christos whether to exercise a particular feature. For instance, a board 496 1.1 christos lacking any I/O devices, or perhaps simply having its I/O devices 497 1.1 christos not wired up, should set `noinferiorio'. 498 1.1 christos 499 1.1 christos Here are the supported board settings: 500 1.1 christos 501 1.1 christos gdb,cannot_call_functions 502 1.1 christos 503 1.1 christos The board does not support inferior call, that is, invoking inferior 504 1.1 christos functions in GDB. 505 1.1 christos 506 1.1 christos gdb,can_reverse 507 1.1 christos 508 1.1 christos The board supports reverse execution. 509 1.1 christos 510 1.1 christos gdb,no_hardware_watchpoints 511 1.1 christos 512 1.1 christos The board does not support hardware watchpoints. 513 1.1 christos 514 1.1 christos gdb,nofileio 515 1.1 christos 516 1.1 christos GDB is unable to intercept target file operations in remote and 517 1.1 christos perform them on the host. 518 1.1 christos 519 1.1 christos gdb,noinferiorio 520 1.1 christos 521 1.1 christos The board is unable to provide I/O capability to the inferior. 522 1.1 christos 523 1.1 christos gdb,noresults 524 1.1 christos 525 1.1 christos A program will not return an exit code or result code (or the value 526 1.1 christos of the result is undefined, and should not be looked at). 527 1.1 christos 528 1.1 christos gdb,nosignals 529 1.1 christos 530 1.1 christos The board does not support signals. 531 1.1 christos 532 1.1 christos gdb,skip_huge_test 533 1.1 christos 534 1.1 christos Skip time-consuming tests on the board with slow connection. 535 1.1 christos 536 1.1 christos gdb,skip_float_tests 537 1.1 christos 538 1.1 christos Skip tests related to floating point. 539 1.1 christos 540 1.1 christos gdb,use_precord 541 1.1 christos 542 1.1 christos The board supports process record. 543 1.1 christos 544 1.3 christos gdb_init_command 545 1.3 christos gdb_init_commands 546 1.3 christos 547 1.3 christos Commands to send to GDB every time a program is about to be run. The 548 1.3 christos first of these settings defines a single command as a string. The 549 1.3 christos second defines a TCL list of commands being a string each. The commands 550 1.3 christos are sent one by one in a sequence, first from `gdb_init_command', if any, 551 1.3 christos followed by individual commands from `gdb_init_command', if any, in this 552 1.3 christos list's order. 553 1.3 christos 554 1.1 christos gdb_server_prog 555 1.1 christos 556 1.1 christos The location of GDBserver. If GDBserver somewhere other than its 557 1.1 christos default location is used in test, specify the location of GDBserver in 558 1.1 christos this variable. The location is a file name for GDBserver, and may be 559 1.1 christos either absolute or relative to the testsuite subdirectory of the build 560 1.1 christos directory. 561 1.1 christos 562 1.1 christos in_proc_agent 563 1.1 christos 564 1.1 christos The location of the in-process agent (used for fast tracepoints and 565 1.1 christos other special tests). If the in-process agent of interest is anywhere 566 1.1 christos other than its default location, set this variable. The location is a 567 1.1 christos filename, and may be either absolute or relative to the testsuite 568 1.1 christos subdirectory of the build directory. 569 1.1 christos 570 1.1 christos noargs 571 1.1 christos 572 1.1 christos GDB does not support argument passing for inferior. 573 1.1 christos 574 1.1 christos no_long_long 575 1.1 christos 576 1.1 christos The board does not support type long long. 577 1.1 christos 578 1.1 christos use_cygmon 579 1.1 christos 580 1.1 christos The board is running the monitor Cygmon. 581 1.1 christos 582 1.1 christos use_gdb_stub 583 1.1 christos 584 1.1 christos The tests are running with a GDB stub. 585 1.1 christos 586 1.1 christos exit_is_reliable 587 1.1 christos 588 1.1 christos Set to true if GDB can assume that letting the program run to end 589 1.1 christos reliably results in program exits being reported as such, as opposed 590 1.1 christos to, e.g., the program ending in an infinite loop or the board 591 1.1 christos crashing/resetting. If not set, this defaults to $use_gdb_stub. In 592 1.1 christos other words, native targets are assumed reliable by default, and 593 1.1 christos remote stubs assumed unreliable. 594 1.1 christos 595 1.1 christos gdb,predefined_tsv 596 1.1 christos 597 1.1 christos The predefined trace state variables the board has. 598 1.1 christos 599 1.6 christos gdb,no_thread_names 600 1.6 christos 601 1.6 christos The target doesn't support thread names. 602 1.1 christos 603 1.9 christos gdb,pie_flag 604 1.9 christos 605 1.9 christos The flag required to force the compiler to produce position-independent 606 1.9 christos executables. 607 1.9 christos 608 1.9 christos gdb,pie_ldflag 609 1.9 christos 610 1.9 christos The flag required to force the linker to produce position-independent 611 1.9 christos executables. 612 1.9 christos 613 1.8 christos gdb,nopie_flag 614 1.8 christos 615 1.8 christos The flag required to force the compiler to produce non-position-independent 616 1.8 christos executables. 617 1.8 christos 618 1.10 christos gdb,nopie_ldflag 619 1.10 christos 620 1.10 christos The flag required to force the linker to produce non-position-independent 621 1.10 christos executables. 622 1.10 christos 623 1.9 christos gdb,debug 624 1.9 christos 625 1.9 christos When set gdb debug is sent to the file gdb.debug in the test output 626 1.9 christos directory. It should be set to a comma separated list of gdb debug 627 1.9 christos components. For example, to turn on debugging for infrun and target, set to 628 1.9 christos "infrun,target". 629 1.9 christos 630 1.9 christos gdbserver,debug 631 1.9 christos 632 1.9 christos When set gdbserver debug is sent to the file gdbserver.debug in the test 633 1.9 christos output directory. For valid values see the entry for GDBSERVER_DEBUG. 634 1.9 christos 635 1.1 christos Testsuite Organization 636 1.1 christos ********************** 637 1.1 christos 638 1.1 christos The testsuite is entirely contained in `gdb/testsuite'. The main 639 1.1 christos directory of the testsuite includes some makefiles and configury, but 640 1.1 christos these are minimal, and used for little besides cleaning up, since the 641 1.1 christos tests themselves handle the compilation of the programs that GDB will 642 1.1 christos run. 643 1.1 christos 644 1.1 christos The file `testsuite/lib/gdb.exp' contains common utility procs useful 645 1.1 christos for all GDB tests, while the directory testsuite/config contains 646 1.1 christos configuration-specific files, typically used for special-purpose 647 1.1 christos definitions of procs like `gdb_load' and `gdb_start'. 648 1.1 christos 649 1.1 christos The tests themselves are to be found in directories named 650 1.1 christos 'testsuite/gdb.* and subdirectories of those. The names of the test 651 1.1 christos files must always end with ".exp". DejaGNU collects the test files by 652 1.1 christos wildcarding in the test directories, so both subdirectories and 653 1.1 christos individual files typically get chosen and run in alphabetical order. 654 1.1 christos 655 1.1 christos The following lists some notable types of subdirectories and what they 656 1.1 christos are for. Since DejaGNU finds test files no matter where they are 657 1.1 christos located, and since each test file sets up its own compilation and 658 1.1 christos execution environment, this organization is simply for convenience and 659 1.1 christos intelligibility. 660 1.1 christos 661 1.1 christos gdb.base 662 1.1 christos 663 1.1 christos This is the base testsuite. The tests in it should apply to all 664 1.1 christos configurations of GDB (but generic native-only tests may live here). 665 1.1 christos The test programs should be in the subset of C that is both valid 666 1.1 christos ANSI/ISO C, and C++. 667 1.1 christos 668 1.1 christos gdb.<lang> 669 1.1 christos 670 1.1 christos Language-specific tests for any language besides C. Examples are 671 1.7 christos gdb.cp for C++ and gdb.rust for Rust. 672 1.1 christos 673 1.1 christos gdb.<platform> 674 1.1 christos 675 1.1 christos Non-portable tests. The tests are specific to a specific 676 1.5 christos configuration (host or target), such as eCos. 677 1.1 christos 678 1.1 christos gdb.arch 679 1.1 christos 680 1.1 christos Architecture-specific tests that are (usually) cross-platform. 681 1.1 christos 682 1.1 christos gdb.<subsystem> 683 1.1 christos 684 1.1 christos Tests that exercise a specific GDB subsystem in more depth. For 685 1.1 christos instance, gdb.disasm exercises various disassemblers, while 686 1.1 christos gdb.stabs tests pathways through the stabs symbol reader. 687 1.1 christos 688 1.1 christos gdb.perf 689 1.1 christos 690 1.1 christos GDB performance tests. 691 1.1 christos 692 1.1 christos Writing Tests 693 1.1 christos ************* 694 1.1 christos 695 1.1 christos In many areas, the GDB tests are already quite comprehensive; you 696 1.1 christos should be able to copy existing tests to handle new cases. Be aware 697 1.1 christos that older tests may use obsolete practices but have not yet been 698 1.1 christos updated. 699 1.1 christos 700 1.1 christos You should try to use `gdb_test' whenever possible, since it includes 701 1.1 christos cases to handle all the unexpected errors that might happen. However, 702 1.1 christos it doesn't cost anything to add new test procedures; for instance, 703 1.1 christos gdb.base/exprs.exp defines a `test_expr' that calls `gdb_test' 704 1.1 christos multiple times. 705 1.1 christos 706 1.1 christos Only use `send_gdb' and `gdb_expect' when absolutely necessary. Even 707 1.1 christos if GDB has several valid responses to a command, you can use 708 1.1 christos `gdb_test_multiple'. Like `gdb_test', `gdb_test_multiple' recognizes 709 1.1 christos internal errors and unexpected prompts. 710 1.1 christos 711 1.1 christos Do not write tests which expect a literal tab character from GDB. On 712 1.1 christos some operating systems (e.g. OpenBSD) the TTY layer expands tabs to 713 1.1 christos spaces, so by the time GDB's output reaches `expect' the tab is gone. 714 1.1 christos 715 1.1 christos The source language programs do *not* need to be in a consistent 716 1.1 christos style. Since GDB is used to debug programs written in many different 717 1.1 christos styles, it's worth having a mix of styles in the testsuite; for 718 1.1 christos instance, some GDB bugs involving the display of source lines might 719 1.1 christos never manifest themselves if the test programs used GNU coding style 720 1.1 christos uniformly. 721 1.1 christos 722 1.1 christos Some testcase results need more detailed explanation: 723 1.1 christos 724 1.1 christos KFAIL 725 1.1 christos 726 1.1 christos Use KFAIL for known problem of GDB itself. You must specify the GDB 727 1.1 christos bug report number, as in these sample tests: 728 1.1 christos 729 1.1 christos kfail "gdb/13392" "continue to marker 2" 730 1.1 christos 731 1.1 christos or 732 1.1 christos 733 1.1 christos setup_kfail gdb/13392 "*-*-*" 734 1.1 christos kfail "continue to marker 2" 735 1.1 christos 736 1.1 christos 737 1.1 christos XFAIL 738 1.1 christos 739 1.1 christos Short for "expected failure", this indicates a known problem with the 740 1.1 christos environment. This could include limitations of the operating system, 741 1.1 christos compiler version, and other components. 742 1.1 christos 743 1.1 christos This example from gdb.base/attach-pie-misread.exp is a sanity check 744 1.1 christos for the target environment: 745 1.1 christos 746 1.1 christos # On x86_64 it is commonly about 4MB. 747 1.1 christos if {$stub_size > 25000000} { 748 1.1 christos xfail "stub size $stub_size is too large" 749 1.1 christos return 750 1.1 christos } 751 1.1 christos 752 1.1 christos You should provide bug report number for the failing component of the 753 1.1 christos environment, if such bug report is available, as with this example 754 1.1 christos referring to a GCC problem: 755 1.1 christos 756 1.1 christos if {[test_compiler_info {gcc-[0-3]-*}] 757 1.1 christos || [test_compiler_info {gcc-4-[0-5]-*}]} { 758 1.1 christos setup_xfail "gcc/46955" *-*-* 759 1.1 christos } 760 1.1 christos gdb_test "python print ttype.template_argument(2)" "&C::c" 761 1.1 christos 762 1.1 christos Note that it is also acceptable, and often preferable, to avoid 763 1.1 christos running the test at all. This is the better option if the limitation 764 1.1 christos is intrinsic to the environment, rather than a bug expected to be 765 1.1 christos fixed in the near future. 766 1.8 christos 767 1.8 christos Local vs Remote vs Native 768 1.8 christos ************************* 769 1.8 christos 770 1.8 christos It's unfortunately easy to get confused in the testsuite about what's 771 1.8 christos native and what's not, what's remote and what's not. The confusion is 772 1.8 christos caused by the overlap in vocabulary between DejaGnu and GDB. 773 1.8 christos 774 1.8 christos From a DejaGnu point of view: 775 1.8 christos 776 1.8 christos - native: the host or target board is considered native if the its 777 1.8 christos triplet is the same as the build system's triplet, 778 1.8 christos 779 1.8 christos - remote: the host or target board is considered remote if it's 780 1.8 christos running on a different machine, and thus require ssh, for example, 781 1.8 christos to run commands, versus simply running commands directly. 782 1.8 christos 783 1.8 christos Note that they are not mutually exclusive, as you can have a remote 784 1.8 christos machine that has the same triplet as the build machine. 785 1.8 christos 786 1.8 christos From a GDB point of view: 787 1.8 christos 788 1.8 christos - native: when GDB uses system calls such as ptrace to interact 789 1.8 christos directly with processes on the same system its running on, 790 1.8 christos 791 1.8 christos - remote: when GDB speaks the RSP (Remote Serial Protocol) with 792 1.8 christos another program doing the ptrace stuff. 793 1.8 christos 794 1.8 christos Note that they are mutually exclusive. An inferior can only be either 795 1.8 christos debugged with the native target, or with the remote target a specific 796 1.8 christos time. 797 1.8 christos 798 1.8 christos That means that there are cases where the target is not remote for 799 1.8 christos DejaGnu, but is remote for GDB (e.g. running GDBserver on the same 800 1.8 christos machine). 801 1.8 christos 802 1.8 christos You can also have a remote target for DejaGnu, but native for GDB 803 1.8 christos (e.g. building on x86 a GDB that runs on ARM and running the 804 1.8 christos testsuite with a remote host). 805 1.8 christos 806 1.8 christos Therefore, care must be taken to check for the right kind of remote. 807 1.8 christos Use [is_remote target] to check whether the DejaGnu target board is 808 1.8 christos remote. When what you really want to know is whether GDB is using the 809 1.8 christos remote protocol, because feature X is only available when GDB debugs 810 1.8 christos natively, check gdb_protocol instead. 811