Home | History | Annotate | Line # | Download | only in testsuite
      1 This is a collection of tests for GDB.
      2 
      3 The file gdb/README contains basic instructions on how to run the
      4 testsuite, while this file documents additional options and controls
      5 that are available.  The GDB wiki may also have some pages with ideas
      6 and suggestions.
      7 
      8 
      9 Running the Testsuite
     10 *********************
     11 
     12 There are two ways to run the testsuite and pass additional parameters
     13 to DejaGnu.  The first is to do `make check' in the main build
     14 directory and specifying the makefile variable `RUNTESTFLAGS':
     15 
     16 	 make check RUNTESTFLAGS='GDB=/usr/bin/gdb gdb.base/a2-run.exp'
     17 
     18 The second is to cd to the testsuite directory and invoke the DejaGnu
     19 `runtest' command directly.
     20 
     21 	cd testsuite
     22 	make site.exp
     23 	runtest GDB=/usr/bin/gdb
     24 
     25 (The `site.exp' file contains a handful of useful variables like host
     26 and target triplets, and pathnames.)
     27 
     28 Parallel testing
     29 ****************
     30 
     31 If not testing with a remote host (in DejaGnu's sense), you can run
     32 the GDB test suite in a fully parallel mode.  In this mode, each .exp
     33 file runs separately and maybe simultaneously.  The test suite ensures
     34 that all the temporary files created by the test suite do not clash,
     35 by putting them into separate directories.  This mode is primarily
     36 intended for use by the Makefile.
     37 
     38 For GNU make, the Makefile tries to run the tests in parallel mode if
     39 any -j option is given.  For a non-GNU make, tests are not
     40 parallelized.
     41 
     42 If RUNTESTFLAGS is not empty, then by default the tests are
     43 serialized.  This can be overridden by either using the
     44 `check-parallel' target in the Makefile, or by setting FORCE_PARALLEL
     45 to any non-empty value:
     46 
     47 	make check-parallel RUNTESTFLAGS="--target_board=native-gdbserver"
     48 	make check RUNTESTFLAGS="--target_board=native-gdbserver" FORCE_PARALLEL=1
     49 
     50 If you want to use runtest directly instead of using the Makefile, see
     51 the description of GDB_PARALLEL below.
     52 
     53 Racy testcases
     54 **************
     55 
     56 Sometimes, new testcases are added to the testsuite that are not
     57 entirely deterministic, and can randomly pass or fail.  We call them
     58 "racy testcases", and they can be bothersome when one is comparing
     59 different testsuite runs.  In order to help identifying them, it is
     60 possible to run the tests several times in a row and ask the testsuite
     61 machinery to analyze the results.  To do that, you need to specify the
     62 RACY_ITER environment variable to make:
     63 
     64 	make check RACY_ITER=5 -j4
     65 
     66 The value assigned to RACY_ITER represents the number of times you
     67 wish to run the tests in sequence (in the example above, the entire
     68 testsuite will be executed 5 times in a row, in parallel).  It is also
     69 possible to check just a specific test:
     70 
     71 	make check TESTS='gdb.base/default.exp' RACY_ITER=3
     72 
     73 One can also decide to call the Makefile rules by hand inside the
     74 gdb/testsuite directory, e.g.:
     75 
     76 	make check-parallel-racy -j4
     77 
     78 In which case the value of the DEFAULT_RACY_ITER variable (inside
     79 gdb/testsuite/Makefile.in) will be used to determine how many
     80 iterations will be run.
     81 
     82 After running the tests, you shall see a file name 'racy.sum' in the
     83 gdb/testsuite directory.  You can also inspect the generated *.log and
     84 *.sum files by looking into the gdb/testsuite/racy_ouputs directory.
     85 
     86 If you already have *.sum files generated from previous testsuite runs
     87 and you would like to analyze them without having to run the testsuite
     88 again, you can also use the 'analyze-racy-logs.py' script directly.
     89 It is located in the gdb/testsuite/ directory, and it expects a list
     90 of two or more *.sum files to be provided as its argument.  For
     91 example:
     92 
     93 	./gdb/testsuite/analyze-racy-logs.py testsuite-01/gdb.sum \
     94 	  testsuite-02/gdb.sum testsuite-03/gdb.sum
     95 
     96 The script will output its analysis report to the standard output.
     97 
     98 Re-running Tests Outside The Testsuite
     99 **************************************
    100 
    101 When running a test, the arguments used to run GDB are saved to gdb.cmd and
    102 all commands sent to GDB are saved to gdb.in.  As well as being a reference
    103 of the commands run, they can be used to manually re-run a test by using
    104 the gdb.in file as a batch file to a GDB launched with the arguments in the
    105 gdb.cmd file, for example:
    106 	$(cat outputs/gdb.base/store/gdb.cmd) -x outputs/gdb.base/store/gdb.in
    107 
    108 Tests that run GDB multiple times will append .1, .2, .3 etc to the end
    109 of each .cmd and .in file.
    110 
    111 When gdbserver is launched as part of a test, a gdbserver.cmd will be created.
    112 To re-run these tests, run the contents of gdbserver.cmd in a separate
    113 terminal before running gdb, for example:
    114 	$(cat outputs/gdb.base/store/gdbserver.cmd)
    115 Alternatively, if the test is run with GDBSERVER_DEBUG="replay", then this
    116 will create a gdbserver.replay file which can be used with the gdbreplay tool,
    117 instead of launching gdbserver.
    118 
    119 Running the Performance Tests
    120 *****************************
    121 
    122 GDB Testsuite includes performance test cases, which are not run together
    123 with other test cases, because performance test cases are slow and need
    124 a quiet system.  There are two ways to run the performance test cases.
    125 The first is to do `make check-perf' in the main build directory:
    126 
    127 	make check-perf RUNTESTFLAGS="solib.exp SOLIB_COUNT=8"
    128 
    129 The second is to cd to the testsuite directory and invoke the DejaGnu
    130 `runtest' command directly.
    131 
    132 	cd testsuite
    133 	make site.exp
    134 	runtest GDB_PERFTEST_MODE=both GDB_PERFTEST_TIMEOUT=4000 --directory=gdb.perf solib.exp SOLIB_COUNT=8
    135 
    136 Only "compile", "run" and "both" are valid to GDB_PERFTEST_MODE.  They
    137 stand for "compile tests only", "run tests only", and "compile and run
    138 tests" respectively.  "both" is the default.  GDB_PERFTEST_TIMEOUT
    139 specify the timeout, which is 3000 in default.  The result of
    140 performance test is appended in `testsuite/perftest.log'.
    141 
    142 Testsuite Parameters
    143 ********************
    144 
    145 The following parameters are DejaGNU variables that you can set to
    146 affect the testsuite run globally.
    147 
    148 GDB
    149 
    150 By default, the testsuite exercises the GDB in the build directory,
    151 but you can set GDB to be a pathname to a different version.  For
    152 instance,
    153 
    154     make check RUNTESTFLAGS=GDB=/usr/bin/gdb
    155 
    156 runs the testsuite on the GDB in /usr/bin.
    157 
    158 GDBSERVER
    159 
    160 You can set GDBSERVER to be a particular GDBserver of interest, so for
    161 instance
    162 
    163     make check RUNTESTFLAGS="GDB=/usr/bin/gdb GDBSERVER=/usr/bin/gdbserver"
    164 
    165 checks both the installed GDB and GDBserver.
    166 
    167 GDB_DATA_DIRECTORY
    168 
    169 If you set GDB, then by default the testsuite assumes you are
    170 exercising an installed GDB, and thus the testsuite lets GDB use its
    171 configured data directory.  Otherwise, if you don't set GDB, then by
    172 default the tested GDB uses the data directory found under the GDB
    173 build directory.  You can override this by setting GDB_DATA_DIRECTORY.
    174 For instance:
    175 
    176     make check \
    177       RUNTESTFLAGS="GDB=/path/to/other/build/gdb \
    178                     GDB_DATA_DIRECTORY=/path/to/other/build/gdb/data-directory"
    179 
    180 INTERNAL_GDBFLAGS
    181 
    182 Command line options passed to all GDB invocations.
    183 The default is set in lib/gdb.exp.
    184 
    185 This is actually considered an internal variable, and you
    186 won't normally want to change it.  However, in some situations,
    187 this may be tweaked as a last resort if the testsuite doesn't
    188 have direct support for the specifics of your environment.
    189 The testsuite does not override a value provided by the user.
    190 
    191 As an example, when testing an installed GDB that has been
    192 configured with `--with-system-gdbinit', like by default,
    193 you do not want ~/.gdbinit to interfere with tests, but, you
    194 may want the system .gdbinit file loaded.  As there's no way to
    195 ask the testsuite, or GDB, to load the system gdbinit but
    196 not ~/.gdbinit, a workaround is then to remove `-nx' from
    197 INTERNAL_GDBFLAGS, and point $HOME at a directory without
    198 a .gdbinit.  For example:
    199 
    200 	cd testsuite
    201 	HOME=`pwd` runtest \
    202 	  GDB=/usr/bin/gdb \
    203 	  GDBSERVER=/usr/bin/gdbserver \
    204 	  INTERNAL_GDBFLAGS="-nw -q -iex 'set height 0' -iex 'set width 0'"
    205 
    206 Note that we do not need to specify '-data-directory' here
    207 as we are testing an installed GDB.
    208 
    209 GDB_PARALLEL
    210 
    211 To use parallel testing mode without using the Makefile, set
    212 GDB_PARALLEL on the runtest command line to "yes".  Before starting
    213 the tests, you must ensure that the directories cache, outputs, and
    214 temp in the test suite build directory are either empty or have been
    215 deleted.  cache in particular is used to share data across invocations
    216 of runtest, and files there may affect the test results.  The Makefile
    217 automatically does these deletions.
    218 
    219 FORCE_PARALLEL
    220 
    221 Setting FORCE_PARALLEL to any non-empty value forces parallel testing
    222 mode even if RUNTESTFLAGS is not empty.
    223 
    224 FORCE_SEPARATE_MI_TTY
    225 
    226 Setting FORCE_MI_SEPARATE_UI to 1 forces all MI testing to start GDB
    227 in console mode, with MI running on a separate TTY, on a secondary UI
    228 started with "new-ui".
    229 
    230 GDB_INOTIFY
    231 
    232 For debugging parallel mode, it is handy to be able to see when a test
    233 case writes to a file outside of its designated output directory.
    234 
    235 If you have the inotify-tools package installed, you can set the
    236 GDB_INOTIFY variable on the runtest command line.  This will cause the
    237 test suite to watch for parallel-unsafe file creations and report
    238 them, both to stdout and in the test suite log file.
    239 
    240 This setting is only meaningful in conjunction with GDB_PARALLEL.
    241 
    242 TESTS
    243 
    244 This variable is used to specify which set of tests to run.
    245 It is passed to make (not runtest) and its contents are a space separated
    246 list of tests to run.
    247 
    248 If using GNU make then the contents are wildcard-expanded using
    249 GNU make's $(wildcard) function.  Test paths must be fully specified,
    250 relative to the "testsuite" subdirectory.  This allows one to run all
    251 tests in a subdirectory by passing "gdb.subdir/*.exp", or more simply
    252 by using the check-gdb.subdir target in the Makefile.
    253 
    254 If for some strange reason one wanted to run all tests that begin with
    255 the letter "d" that is also possible: TESTS="*/d*.exp".
    256 
    257 Do not write */*.exp to specify all tests (assuming all tests are only
    258 nested one level deep, which is not necessarily true).  This will pick up
    259 .exp files in ancillary directories like "lib" and "config".
    260 Instead write gdb.*/*.exp.
    261 
    262 Example:
    263 
    264 	make -j10 check TESTS="gdb.server/[s-w]*.exp */x*.exp"
    265 
    266 If not using GNU make then the value is passed directly to runtest.
    267 If not specified, all tests are run.
    268 
    269 READ1
    270 
    271 This make (not runtest) variable is used to specify whether the
    272 testsuite preloads the read1.so library into expect.  Any non-empty
    273 value means true.  See "Race detection" below.
    274 
    275 GDB_TEST_SOCKETHOST
    276 
    277 This variable can provide the hostname/address that should be used
    278 when performing GDBserver-related tests.  This is useful in some
    279 situations, e.g., when you want to test the IPv6 connectivity of GDB
    280 and GDBserver, or when using a different hostname/address is needed.
    281 For example, to make GDB and GDBserver use IPv6-only connections, you
    282 can do:
    283 
    284 	make check TESTS="gdb.server/*.exp" RUNTESTFLAGS='GDB_TEST_SOCKETHOST=tcp6:[::1]'
    285 
    286 Note that only a hostname/address can be provided, without a port
    287 number.
    288 
    289 TS
    290 
    291 This variable turns on the timestamp printing for each line of "make
    292 check".  Note that the timestamp will be printed on stdout output
    293 only.  In other words, there will be no timestamp output on either
    294 gdb.sum and gdb.log files.  If you would like to enable timestamp
    295 printing, you can do:
    296 
    297 	make check TS=1
    298 
    299 TS_FORMAT
    300 
    301 You can provide a custom format for timestamp printing with this
    302 variable.  The format must be a string compatible with "strftime".
    303 This variable is only useful when the TS variable is also provided.
    304 If you would like to change the output format of the timestamp, you
    305 can do:
    306 
    307 	make check TS=1 TS_FORMAT='[%b %H:%S]'
    308 
    309 GDB_DEBUG
    310 
    311 When set gdb debug is sent to the file gdb.debug in the test output
    312 directory.  It should be set to a comma separated list of gdb debug
    313 components.
    314 For example, to turn on debugging for infrun and target, you can do:
    315 
    316 	make check GDB_DEBUG="infrun,target"
    317 
    318 GDBSERVER_DEBUG
    319 
    320 When set gdbserver debug is sent to the a file in the test output directory.
    321 It should be set to a comma separated list of the following options:
    322 	debug  - write gdbserver debug to gdbserver.debug.
    323 	remote - write gdbserver remote debug to gdbserver.debug.
    324 	replay - write a replay log to the file gdbserver.replay for use
    325 		 with gdbreplay.
    326 Alternatively, it can be set to "all" to turn on all the above
    327 For example, to turn on gdbserver debugging, you can do:
    328 
    329 	make check GDBSERVER_DEBUG="debug,replay"
    330 
    331 GDB_TARGET_USERNAME
    332 GDB_HOST_USERNAME
    333 
    334 These settings are only used with the check-all-boards target, and
    335 should be the usernames of two separate users on the local machine,
    336 both of which the current user can ssh to without a password.
    337 
    338 These users will be used by board files that simulate remote targets
    339 by switching to a different user on the same machine.  These users
    340 will have random files copied into their $HOME directories, so it is a
    341 good idea to setup new users just for this purpose.
    342 
    343 Testing All Simple Boards
    344 *************************
    345 
    346 There are a number of boards that ship with GDB that simulate common
    347 debug scenarios.  For example by sshing to a different user on the
    348 local machine and running gdbserver as this alternative user we aim to
    349 simulate a true remote debug experience.
    350 
    351 There is a script binutils-gdb/gdb/testssuite/make-check-all.sh which
    352 can be used to run a defined set of tests using all of the available
    353 simple board files.  Support for using this script is also included in
    354 GDB's makefile, and can be used as:
    355 
    356   make check-all-boards GDB_TARGET_USERNAME=remote-target \
    357 			GDB_HOST_USERNAME=remote-host \
    358 			TESTS="gdb.base/break.exp"
    359 
    360 The 'remote-target' and 'remote-host' can be replaced with any user
    361 names on the local machine, the only requirements are that the current
    362 user must be able to ssh to these users without a password, and these
    363 users must be happy to have arbitrary files copied into their $HOME
    364 directory.  Ideally, these users would be setup just for GDB testing.
    365 
    366 The check-all-boards target requires that TESTS be defined, though it
    367 is fine to include multiple tests.
    368 
    369 The results are preserved, and can be found in the directory
    370 gdb/testsuite/check-all/.  The results are split by the board file
    371 used.
    372 
    373 Architecture-specific Parameters
    374 ******************************
    375 
    376 This section documents architecture-specific parameters that can be used with
    377 the GDB testsuite.
    378 
    379 - AArch64 (Linux)
    380 
    381 ARM_CC_FOR_TARGET
    382 
    383 The AArch64 ports of GDB and GDBserver support debugging AArch32
    384 32-bit programs running on 64-bit state.  There are some tests under
    385 gdb.multi/ that exercise this particular feature.
    386 
    387 By default, the testsuite tries to find a compiler capable of
    388 generating 32-bit executables.  If no compiler is found, or if the
    389 32-bit executable generated by the found compiler can't be executed
    390 correctly, the tests will be marked UNSUPPORTED.  The list of 32-bit
    391 Arm compiler names the testsuite will try can be found in
    392 gdb/testsuite/lib/gdb.exp:arm_cc_for_target.
    393 
    394 You can set ARM_CC_FOR_TARGET to override the search and explicitly
    395 specify the compiler to use.  This variable should contain the command
    396 line for the compiler, including the full path to it, if the compiler
    397 is not in $PATH.
    398 
    399 Example:
    400 
    401 	make check-gdb TESTS="gdb.multi/multi-arch.exp" RUNTESTFLAGS="ARM_CC_FOR_TARGET=arm-linux-gnueabihf-gcc"
    402 
    403 Race detection
    404 **************
    405 
    406 The testsuite includes a mechanism that helps detect test races.
    407 
    408 For example, say the program running under expect outputs "abcd", and
    409 a test does something like this:
    410 
    411   expect {
    412     "a.*c" {
    413     }
    414     "b" {
    415     }
    416     "a" {
    417     }
    418   }
    419 
    420 Which case happens to match depends on what expect manages to read
    421 into its internal buffer in one go.  If it manages to read three bytes
    422 or more, then the first case matches.  If it manages to read two
    423 bytes, then the second case matches.  If it manages to read only one
    424 byte, then the third case matches.
    425 
    426 To help detect these cases, the race detection mechanism preloads a
    427 library into expect that forces the `read' system call to always
    428 return at most 1 byte.
    429 
    430 To enable this, either pass a non-empty value in the READ1 make
    431 variable, or use the check-read1 make target instead of check.
    432 
    433 Example:
    434 
    435 	make -j10 check-read1 TESTS="*/paginate-*.exp"
    436 
    437 If you've already built the read1 support code, either via a previous
    438 'check-read1' run, or by using "make read1", you can use:
    439 
    440 	make -j10 check READ1="1"
    441 
    442 Note: While the intention is to detect races and make otherwise passing tests
    443 fail, it can also have the effect of making otherwise failing tests pass.
    444 This happens f.i. if the test is trying to match a gdb prompt using an end of
    445 input marker "${gdb_prompt} $" and there is output after the gdb prompt.  This
    446 may either pass or fail in normal operation, but using check-read1 will ensure
    447 that it passes.  Use check-readmore to detect this type of failure.
    448 
    449 Testsuite Configuration
    450 ***********************
    451 
    452 It is possible to adjust the behavior of the testsuite by defining
    453 the global variables listed below, either in a `site.exp' file,
    454 or in a board file.
    455 
    456 gdb_test_timeout
    457 
    458 Defining this variable changes the default timeout duration used
    459 during communication with GDB.  More specifically, the global variable
    460 used during testing is `timeout', but this variable gets reset to
    461 `gdb_test_timeout' at the beginning of each testcase, which ensures
    462 that any local change to `timeout' in a testcase does not affect
    463 subsequent testcases.
    464 
    465 This global variable comes in handy when the debugger is slower than
    466 normal due to the testing environment, triggering unexpected `TIMEOUT'
    467 test failures.  Examples include when testing on a remote machine, or
    468 against a system where communications are slow.
    469 
    470 If not specifically defined, this variable gets automatically defined
    471 to the same value as `timeout' during the testsuite initialization.
    472 The default value of the timeout is defined in the file
    473 `testsuite/config/unix.exp' (at least for Unix hosts; board files may
    474 have their own values).
    475 
    476 gdb_reverse_timeout
    477 
    478 Defining this variable changes the default timeout duration when tests
    479 under gdb.reverse directory are running.  Process record and reverse
    480 debugging is so slow that its tests have unexpected `TIMEOUT' test
    481 failures.  This global variable is useful to bump up the value of
    482 `timeout' for gdb.reverse tests and doesn't cause any delay where
    483 actual failures happen in the rest of the testsuite.
    484 
    485 
    486 Board Settings
    487 **************
    488 
    489 DejaGNU includes the concept of a "board file", which specifies
    490 testing details for a particular target (which are often bare circuit
    491 boards, thus the name).
    492 
    493 In the GDB testsuite specifically, the board file may include a
    494 number of "board settings" that test cases may check before deciding
    495 whether to exercise a particular feature.  For instance, a board
    496 lacking any I/O devices, or perhaps simply having its I/O devices
    497 not wired up, should set `noinferiorio'.
    498 
    499 Here are the supported board settings:
    500 
    501 gdb,cannot_call_functions
    502 
    503   The board does not support inferior call, that is, invoking inferior
    504   functions in GDB.
    505 
    506 gdb,can_reverse
    507 
    508   The board supports reverse execution.
    509 
    510 gdb,no_hardware_watchpoints
    511 
    512   The board does not support hardware watchpoints.
    513 
    514 gdb,nofileio
    515 
    516   GDB is unable to intercept target file operations in remote and
    517   perform them on the host.
    518 
    519 gdb,noinferiorio
    520 
    521   The board is unable to provide I/O capability to the inferior.
    522 
    523 gdb,noresults
    524 
    525   A program will not return an exit code or result code (or the value
    526   of the result is undefined, and should not be looked at).
    527 
    528 gdb,nosignals
    529 
    530   The board does not support signals.
    531 
    532 gdb,skip_huge_test
    533 
    534   Skip time-consuming tests on the board with slow connection.
    535 
    536 gdb,skip_float_tests
    537 
    538   Skip tests related to floating point.
    539 
    540 gdb,use_precord
    541 
    542   The board supports process record.
    543 
    544 gdb_init_command
    545 gdb_init_commands
    546 
    547   Commands to send to GDB every time a program is about to be run.  The
    548   first of these settings defines a single command as a string.  The
    549   second defines a TCL list of commands being a string each.  The commands
    550   are sent one by one in a sequence, first from `gdb_init_command', if any,
    551   followed by individual commands from `gdb_init_command', if any, in this
    552   list's order.
    553 
    554 gdb_server_prog
    555 
    556   The location of GDBserver.  If GDBserver somewhere other than its
    557   default location is used in test, specify the location of GDBserver in
    558   this variable.  The location is a file name for GDBserver, and may be
    559   either absolute or relative to the testsuite subdirectory of the build
    560   directory.
    561 
    562 in_proc_agent
    563 
    564   The location of the in-process agent (used for fast tracepoints and
    565   other special tests).  If the in-process agent of interest is anywhere
    566   other than its default location, set this variable.  The location is a
    567   filename, and may be either absolute or relative to the testsuite
    568   subdirectory of the build directory.
    569 
    570 noargs
    571 
    572   GDB does not support argument passing for inferior.
    573 
    574 no_long_long
    575 
    576   The board does not support type long long.
    577 
    578 use_cygmon
    579 
    580   The board is running the monitor Cygmon.
    581 
    582 use_gdb_stub
    583 
    584   The tests are running with a GDB stub.
    585 
    586 exit_is_reliable
    587 
    588   Set to true if GDB can assume that letting the program run to end
    589   reliably results in program exits being reported as such, as opposed
    590   to, e.g., the program ending in an infinite loop or the board
    591   crashing/resetting.  If not set, this defaults to $use_gdb_stub.  In
    592   other words, native targets are assumed reliable by default, and
    593   remote stubs assumed unreliable.
    594 
    595 gdb,predefined_tsv
    596 
    597   The predefined trace state variables the board has.
    598 
    599 gdb,no_thread_names
    600 
    601   The target doesn't support thread names.
    602 
    603 gdb,pie_flag
    604 
    605   The flag required to force the compiler to produce position-independent
    606   executables.
    607 
    608 gdb,pie_ldflag
    609 
    610   The flag required to force the linker to produce position-independent
    611   executables.
    612 
    613 gdb,nopie_flag
    614 
    615   The flag required to force the compiler to produce non-position-independent
    616   executables.
    617 
    618 gdb,nopie_ldflag
    619 
    620   The flag required to force the linker to produce non-position-independent
    621   executables.
    622 
    623 gdb,debug
    624 
    625   When set gdb debug is sent to the file gdb.debug in the test output
    626   directory.  It should be set to a comma separated list of gdb debug
    627   components. For example, to turn on debugging for infrun and target, set to
    628   "infrun,target".
    629 
    630 gdbserver,debug
    631 
    632   When set gdbserver debug is sent to the file gdbserver.debug in the test
    633   output directory.  For valid values see the entry for GDBSERVER_DEBUG.
    634 
    635 Testsuite Organization
    636 **********************
    637 
    638 The testsuite is entirely contained in `gdb/testsuite'.  The main
    639 directory of the testsuite includes some makefiles and configury, but
    640 these are minimal, and used for little besides cleaning up, since the
    641 tests themselves handle the compilation of the programs that GDB will
    642 run.
    643 
    644 The file `testsuite/lib/gdb.exp' contains common utility procs useful
    645 for all GDB tests, while the directory testsuite/config contains
    646 configuration-specific files, typically used for special-purpose
    647 definitions of procs like `gdb_load' and `gdb_start'.
    648 
    649 The tests themselves are to be found in directories named
    650 'testsuite/gdb.* and subdirectories of those.  The names of the test
    651 files must always end with ".exp".  DejaGNU collects the test files by
    652 wildcarding in the test directories, so both subdirectories and
    653 individual files typically get chosen and run in alphabetical order.
    654 
    655 The following lists some notable types of subdirectories and what they
    656 are for.  Since DejaGNU finds test files no matter where they are
    657 located, and since each test file sets up its own compilation and
    658 execution environment, this organization is simply for convenience and
    659 intelligibility.
    660 
    661 gdb.base
    662 
    663 This is the base testsuite.  The tests in it should apply to all
    664 configurations of GDB (but generic native-only tests may live here).
    665 The test programs should be in the subset of C that is both valid
    666 ANSI/ISO C, and C++.
    667 
    668 gdb.<lang>
    669 
    670 Language-specific tests for any language besides C.  Examples are
    671 gdb.cp for C++ and gdb.rust for Rust.
    672 
    673 gdb.<platform>
    674 
    675 Non-portable tests.  The tests are specific to a specific
    676 configuration (host or target), such as eCos.
    677 
    678 gdb.arch
    679 
    680 Architecture-specific tests that are (usually) cross-platform.
    681 
    682 gdb.<subsystem>
    683 
    684 Tests that exercise a specific GDB subsystem in more depth.  For
    685 instance, gdb.disasm exercises various disassemblers, while
    686 gdb.stabs tests pathways through the stabs symbol reader.
    687 
    688 gdb.perf
    689 
    690 GDB performance tests.
    691 
    692 Writing Tests
    693 *************
    694 
    695 In many areas, the GDB tests are already quite comprehensive; you
    696 should be able to copy existing tests to handle new cases.  Be aware
    697 that older tests may use obsolete practices but have not yet been
    698 updated.
    699 
    700 You should try to use `gdb_test' whenever possible, since it includes
    701 cases to handle all the unexpected errors that might happen.  However,
    702 it doesn't cost anything to add new test procedures; for instance,
    703 gdb.base/exprs.exp defines a `test_expr' that calls `gdb_test'
    704 multiple times.
    705 
    706 Only use `send_gdb' and `gdb_expect' when absolutely necessary.  Even
    707 if GDB has several valid responses to a command, you can use
    708 `gdb_test_multiple'.  Like `gdb_test', `gdb_test_multiple' recognizes
    709 internal errors and unexpected prompts.
    710 
    711 Do not write tests which expect a literal tab character from GDB.  On
    712 some operating systems (e.g. OpenBSD) the TTY layer expands tabs to
    713 spaces, so by the time GDB's output reaches `expect' the tab is gone.
    714 
    715 The source language programs do *not* need to be in a consistent
    716 style.  Since GDB is used to debug programs written in many different
    717 styles, it's worth having a mix of styles in the testsuite; for
    718 instance, some GDB bugs involving the display of source lines might
    719 never manifest themselves if the test programs used GNU coding style
    720 uniformly.
    721 
    722 Some testcase results need more detailed explanation:
    723 
    724 KFAIL
    725 
    726 Use KFAIL for known problem of GDB itself.  You must specify the GDB
    727 bug report number, as in these sample tests:
    728 
    729 	kfail "gdb/13392" "continue to marker 2"
    730 
    731 or
    732 
    733 	setup_kfail gdb/13392 "*-*-*"
    734 	kfail "continue to marker 2"
    735 
    736 
    737 XFAIL
    738 
    739 Short for "expected failure", this indicates a known problem with the
    740 environment.  This could include limitations of the operating system,
    741 compiler version, and other components.
    742 
    743 This example from gdb.base/attach-pie-misread.exp is a sanity check
    744 for the target environment:
    745 
    746 	# On x86_64 it is commonly about 4MB.
    747 	if {$stub_size > 25000000} {
    748 	    xfail "stub size $stub_size is too large"
    749 	    return
    750 	}
    751 
    752 You should provide bug report number for the failing component of the
    753 environment, if such bug report is available, as with this example
    754 referring to a GCC problem:
    755 
    756 	  if {[test_compiler_info {gcc-[0-3]-*}]
    757 	      || [test_compiler_info {gcc-4-[0-5]-*}]} {
    758 	      setup_xfail "gcc/46955" *-*-*
    759 	  }
    760 	  gdb_test "python print ttype.template_argument(2)" "&C::c"
    761 
    762 Note that it is also acceptable, and often preferable, to avoid
    763 running the test at all.  This is the better option if the limitation
    764 is intrinsic to the environment, rather than a bug expected to be
    765 fixed in the near future.
    766 
    767 Local vs Remote vs Native
    768 *************************
    769 
    770 It's unfortunately easy to get confused in the testsuite about what's
    771 native and what's not, what's remote and what's not.  The confusion is
    772 caused by the overlap in vocabulary between DejaGnu and GDB.
    773 
    774 From a DejaGnu point of view:
    775 
    776  - native: the host or target board is considered native if the its
    777    triplet is the same as the build system's triplet,
    778 
    779  - remote: the host or target board is considered remote if it's
    780    running on a different machine, and thus require ssh, for example,
    781    to run commands, versus simply running commands directly.
    782 
    783 Note that they are not mutually exclusive, as you can have a remote
    784 machine that has the same triplet as the build machine.
    785 
    786 From a GDB point of view:
    787 
    788  - native: when GDB uses system calls such as ptrace to interact
    789    directly with processes on the same system its running on,
    790 
    791  - remote: when GDB speaks the RSP (Remote Serial Protocol) with
    792    another program doing the ptrace stuff.
    793 
    794 Note that they are mutually exclusive.  An inferior can only be either
    795 debugged with the native target, or with the remote target a specific
    796 time.
    797 
    798 That means that there are cases where the target is not remote for
    799 DejaGnu, but is remote for GDB (e.g. running GDBserver on the same
    800 machine).
    801 
    802 You can also have a remote target for DejaGnu, but native for GDB
    803 (e.g.  building on x86 a GDB that runs on ARM and running the
    804 testsuite with a remote host).
    805 
    806 Therefore, care must be taken to check for the right kind of remote.
    807 Use [is_remote target] to check whether the DejaGnu target board is
    808 remote.  When what you really want to know is whether GDB is using the
    809 remote protocol, because feature X is only available when GDB debugs
    810 natively, check gdb_protocol instead.
    811