1 <!DOCTYPE html> 2 <html> 3 <!-- Created by GNU Texinfo 7.2, https://www.gnu.org/software/texinfo/ --> 4 <head> 5 <meta http-equiv="Content-Type" content="text/html; charset=utf-8"> 6 <!-- Copyright 1988-2024 Free Software Foundation, Inc. 7 8 Permission is granted to copy, distribute and/or modify this document 9 under the terms of the GNU Free Documentation License, Version 1.3 or 10 any later version published by the Free Software Foundation; with no 11 Invariant Sections, the Front-Cover texts being (a) (see below), and 12 with the Back-Cover Texts being (b) (see below). A copy of the 13 license is included in the section entitled "GNU 14 Free Documentation License". 15 16 (a) The FSF's Front-Cover Text is: 17 18 A GNU Manual 19 20 (b) The FSF's Back-Cover Text is: 21 22 You have freedom to copy and modify this GNU Manual, like GNU 23 software. Copies published by the Free Software Foundation raise 24 funds for GNU development. --> 25 <title>Installing GCC: Testing</title> 26 27 <meta name="description" content="Installing GCC: Testing"> 28 <meta name="keywords" content="Installing GCC: Testing"> 29 <meta name="resource-type" content="document"> 30 <meta name="distribution" content="global"> 31 <meta name="Generator" content="makeinfo"> 32 <meta name="viewport" content="width=device-width,initial-scale=1"> 33 34 <style type="text/css"> 35 <!-- 36 a.copiable-link {visibility: hidden; text-decoration: none; line-height: 0em} 37 div.example {margin-left: 3.2em} 38 span:hover a.copiable-link {visibility: visible} 39 ul.mark-bullet {list-style-type: disc} 40 --> 41 </style> 42 43 44 </head> 45 46 <body lang="en"> 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 <a class="index-entry-id" id="index-Testing"></a> 72 <a class="index-entry-id" id="index-Installing-GCC_003a-Testing"></a> 73 <a class="index-entry-id" id="index-Testsuite"></a> 74 75 <p>Before you install GCC, we encourage you to run the testsuites and to 76 compare your results with results from a similar configuration that have 77 been submitted to the 78 <a class="uref" href="https://gcc.gnu.org/ml/gcc-testresults/">gcc-testresults mailing list</a>. 79 This step is optional and may require you to download additional software, 80 but it can give you confidence in your new GCC installation or point out 81 problems before you install and start using your new GCC. 82 </p> 83 <p>First, you must have <a class="uref" href="download.html">downloaded the testsuites</a>. 84 These are included in the source tarball. 85 </p> 86 <p>Second, you must have the testing tools installed. This includes 87 <a class="uref" href="https://www.gnu.org/software/dejagnu/">DejaGnu</a>, Tcl, and Expect; 88 the DejaGnu site has links to these. 89 Some optional tests also require Python3 and pytest module. 90 </p> 91 <p>If the directories where <code class="command">runtest</code> and <code class="command">expect</code> were 92 installed are not in the <code class="env">PATH</code>, you may need to set the following 93 environment variables appropriately, as in the following example (which 94 assumes that DejaGnu has been installed under <samp class="file">/usr/local</samp>): 95 </p> 96 <div class="example smallexample"> 97 <pre class="example-preformatted">TCL_LIBRARY = /usr/local/share/tcl8.0 98 DEJAGNULIBS = /usr/local/share/dejagnu 99 </pre></div> 100 101 <p>(On systems such as Cygwin, these paths are required to be actual 102 paths, not mounts or links; presumably this is due to some lack of 103 portability in the DejaGnu code.) 104 </p> 105 106 <p>Finally, you can run the testsuite (which may take a long time): 107 </p><div class="example smallexample"> 108 <pre class="example-preformatted">cd <var class="var">objdir</var>; make -k check 109 </pre></div> 110 111 <p>This will test various components of GCC, such as compiler 112 front ends and runtime libraries. While running the testsuite, DejaGnu 113 might emit some harmless messages resembling 114 ‘<samp class="samp">WARNING: Couldn't find the global config file.</samp>’ or 115 ‘<samp class="samp">WARNING: Couldn't find tool init file</samp>’ that can be ignored. 116 </p> 117 <p>If you are testing a cross-compiler, you may want to run the testsuite 118 on a simulator as described at <a class="uref" href="https://gcc.gnu.org/simtest-howto.html">https://gcc.gnu.org/simtest-howto.html</a>. 119 </p> 120 <div class="section-level-extent" id="How-can-you-run-the-testsuite-on-selected-tests_003f"> 121 <h3 class="section"><span>How can you run the testsuite on selected tests?<a class="copiable-link" href="#How-can-you-run-the-testsuite-on-selected-tests_003f"> ¶</a></span></h3> 122 123 <p>In order to run sets of tests selectively, there are targets 124 ‘<samp class="samp">make check-gcc</samp>’ and language specific ‘<samp class="samp">make check-c</samp>’, 125 ‘<samp class="samp">make check-c++</samp>’, ‘<samp class="samp">make check-d</samp>’ ‘<samp class="samp">make check-fortran</samp>’, 126 ‘<samp class="samp">make check-ada</samp>’, ‘<samp class="samp">make check-m2</samp>’, ‘<samp class="samp">make check-objc</samp>’, 127 ‘<samp class="samp">make check-obj-c++</samp>’, ‘<samp class="samp">make check-lto</samp>’ in the <samp class="file">gcc</samp> 128 subdirectory of the object directory. You can also just run 129 ‘<samp class="samp">make check</samp>’ in a subdirectory of the object directory. 130 </p> 131 132 <p>A more selective way to just run all <code class="command">gcc</code> execute tests in the 133 testsuite is to use 134 </p> 135 <div class="example smallexample"> 136 <pre class="example-preformatted">make check-gcc RUNTESTFLAGS="execute.exp <var class="var">other-options</var>" 137 </pre></div> 138 139 <p>Likewise, in order to run only the <code class="command">g++</code> “old-deja” tests in 140 the testsuite with filenames matching ‘<samp class="samp">9805*</samp>’, you would use 141 </p> 142 <div class="example smallexample"> 143 <pre class="example-preformatted">make check-g++ RUNTESTFLAGS="old-deja.exp=9805* <var class="var">other-options</var>" 144 </pre></div> 145 146 <p>The file-matching expression following <var class="var">filename</var><code class="command">.exp=</code> is treated 147 as a series of whitespace-delimited glob expressions so that multiple patterns 148 may be passed, although any whitespace must either be escaped or surrounded by 149 single quotes if multiple expressions are desired. For example, 150 </p> 151 <div class="example smallexample"> 152 <pre class="example-preformatted">make check-g++ RUNTESTFLAGS="old-deja.exp=9805*\ virtual2.c <var class="var">other-options</var>" 153 make check-g++ RUNTESTFLAGS="'old-deja.exp=9805* virtual2.c' <var class="var">other-options</var>" 154 </pre></div> 155 156 <p>The <samp class="file">*.exp</samp> files are located in the testsuite directories of the GCC 157 source, the most important ones being <samp class="file">compile.exp</samp>, 158 <samp class="file">execute.exp</samp>, <samp class="file">dg.exp</samp> and <samp class="file">old-deja.exp</samp>. 159 To get a list of the possible <samp class="file">*.exp</samp> files, pipe the 160 output of ‘<samp class="samp">make check</samp>’ into a file and look at the 161 ‘<samp class="samp">Running … .exp</samp>’ lines. 162 </p> 163 </div> 164 <div class="section-level-extent" id="Passing-options-and-running-multiple-testsuites"> 165 <h3 class="section"><span>Passing options and running multiple testsuites<a class="copiable-link" href="#Passing-options-and-running-multiple-testsuites"> ¶</a></span></h3> 166 167 <p>You can pass multiple options to the testsuite using the 168 ‘<samp class="samp">--target_board</samp>’ option of DejaGNU, either passed as part of 169 ‘<samp class="samp">RUNTESTFLAGS</samp>’, or directly to <code class="command">runtest</code> if you prefer to 170 work outside the makefiles. For example, 171 </p> 172 <div class="example smallexample"> 173 <pre class="example-preformatted">make check-g++ RUNTESTFLAGS="--target_board=unix/-O3/-fmerge-constants" 174 </pre></div> 175 176 <p>will run the standard <code class="command">g++</code> testsuites (“unix” is the target name 177 for a standard native testsuite situation), passing 178 ‘<samp class="samp">-O3 -fmerge-constants</samp>’ to the compiler on every test, i.e., 179 slashes separate options. 180 </p> 181 <p>You can run the testsuites multiple times using combinations of options 182 with a syntax similar to the brace expansion of popular shells: 183 </p> 184 <div class="example smallexample"> 185 <pre class="example-preformatted">..."--target_board=arm-sim\{-mhard-float,-msoft-float\}\{-O1,-O2,-O3,\}" 186 </pre></div> 187 188 <p>(Note the empty option caused by the trailing comma in the final group.) 189 The following will run each testsuite eight times using the ‘<samp class="samp">arm-sim</samp>’ 190 target, as if you had specified all possible combinations yourself: 191 </p> 192 <div class="example smallexample"> 193 <pre class="example-preformatted">--target_board='arm-sim/-mhard-float/-O1 \ 194 arm-sim/-mhard-float/-O2 \ 195 arm-sim/-mhard-float/-O3 \ 196 arm-sim/-mhard-float \ 197 arm-sim/-msoft-float/-O1 \ 198 arm-sim/-msoft-float/-O2 \ 199 arm-sim/-msoft-float/-O3 \ 200 arm-sim/-msoft-float' 201 </pre></div> 202 203 <p>They can be combined as many times as you wish, in arbitrary ways. This 204 list: 205 </p> 206 <div class="example smallexample"> 207 <pre class="example-preformatted">..."--target_board=unix/-Wextra\{-O3,-fno-strength\}\{-fomit-frame,\}" 208 </pre></div> 209 210 <p>will generate four combinations, all involving ‘<samp class="samp">-Wextra</samp>’. 211 </p> 212 <p>The disadvantage to this method is that the testsuites are run in serial, 213 which is a waste on multiprocessor systems. For users with GNU Make and 214 a shell which performs brace expansion, you can run the testsuites in 215 parallel by having the shell perform the combinations and <code class="command">make</code> 216 do the parallel runs. Instead of using ‘<samp class="samp">--target_board</samp>’, use a 217 special makefile target: 218 </p> 219 <div class="example smallexample"> 220 <pre class="example-preformatted">make -j<var class="var">N</var> check-<var class="var">testsuite</var>//<var class="var">test-target</var>/<var class="var">option1</var>/<var class="var">option2</var>/... 221 </pre></div> 222 223 <p>For example, 224 </p> 225 <div class="example smallexample"> 226 <pre class="example-preformatted">make -j3 check-gcc//sh-hms-sim/{-m1,-m2,-m3,-m3e,-m4}/{,-nofpu} 227 </pre></div> 228 229 <p>will run three concurrent “make-gcc” testsuites, eventually testing all 230 ten combinations as described above. Note that this is currently only 231 supported in the <samp class="file">gcc</samp> subdirectory. (To see how this works, try 232 typing <code class="command">echo</code> before the example given here.) 233 </p> 234 235 </div> 236 <div class="section-level-extent" id="How-to-interpret-test-results"> 237 <h3 class="section"><span>How to interpret test results<a class="copiable-link" href="#How-to-interpret-test-results"> ¶</a></span></h3> 238 239 <p>The result of running the testsuite are various <samp class="file">*.sum</samp> and <samp class="file">*.log</samp> 240 files in the testsuite subdirectories. The <samp class="file">*.log</samp> files contain a 241 detailed log of the compiler invocations and the corresponding 242 results, the <samp class="file">*.sum</samp> files summarize the results. These summaries 243 contain status codes for all tests: 244 </p> 245 <ul class="itemize mark-bullet"> 246 <li>PASS: the test passed as expected 247 </li><li>XPASS: the test unexpectedly passed 248 </li><li>FAIL: the test unexpectedly failed 249 </li><li>XFAIL: the test failed as expected 250 </li><li>UNSUPPORTED: the test is not supported on this platform 251 </li><li>ERROR: the testsuite detected an error 252 </li><li>WARNING: the testsuite detected a possible problem 253 </li></ul> 254 255 <p>It is normal for some tests to report unexpected failures. At the 256 current time the testing harness does not allow fine grained control 257 over whether or not a test is expected to fail. This problem should 258 be fixed in future releases. 259 </p> 260 261 </div> 262 <div class="section-level-extent" id="Submitting-test-results"> 263 <h3 class="section"><span>Submitting test results<a class="copiable-link" href="#Submitting-test-results"> ¶</a></span></h3> 264 265 <p>If you want to report the results to the GCC project, use the 266 <samp class="file">contrib/test_summary</samp> shell script. Start it in the <var class="var">objdir</var> with 267 </p> 268 <div class="example smallexample"> 269 <pre class="example-preformatted"><var class="var">srcdir</var>/contrib/test_summary -p your_commentary.txt \ 270 -m gcc-testresults (a] gcc.gnu.org |sh 271 </pre></div> 272 273 <p>This script uses the <code class="command">Mail</code> program to send the results, so 274 make sure it is in your <code class="env">PATH</code>. The file <samp class="file">your_commentary.txt</samp> is 275 prepended to the testsuite summary and should contain any special 276 remarks you have on your results or your build environment. Please 277 do not edit the testsuite result block or the subject line, as these 278 messages may be automatically processed. 279 </p> 280 <hr /> 281 <p> 282 <p><a class="uref" href="./index.html">Return to the GCC Installation page</a> 283 </p> 284 285 286 287 288 289 </div> 290 291 292 293 </body> 294 </html> 295
Indexes created Mon Oct 26 00:26:21 UTC 2026