NEWS revision 1.1.1.9.2.3
1 1.1.1.5 jmmv Major changes between releases Automated Testing Framework 2 1.1.1.5 jmmv =========================================================================== 3 1.1 jmmv 4 1.1 jmmv 5 1.1.1.9.2.3 yamt Changes in version 0.20 6 1.1.1.9.2.3 yamt *********************** 7 1.1.1.9.2.3 yamt 8 1.1.1.9.2.3 yamt Experimental version released on February 7th, 2014. 9 1.1.1.9.2.3 yamt 10 1.1.1.9.2.3 yamt This is the first release without the code for the deprecated tools. If 11 1.1.1.9.2.3 yamt you require such code, please fetch a copy of the 0.19 release and extract 12 1.1.1.9.2.3 yamt the 'tools' directory for your own consumption. 13 1.1.1.9.2.3 yamt 14 1.1.1.9.2.3 yamt * Removed the deprecated tools. This includes atf-config, atf-report, 15 1.1.1.9.2.3 yamt atf-run and atf-version. 16 1.1.1.9.2.3 yamt 17 1.1.1.9.2.3 yamt 18 1.1.1.9.2.3 yamt Changes in version 0.19 19 1.1.1.9.2.3 yamt *********************** 20 1.1.1.9.2.3 yamt 21 1.1.1.9.2.3 yamt Experimental version released on February 7th, 2014. 22 1.1.1.9.2.3 yamt 23 1.1.1.9.2.3 yamt This is the last release to bundle the code for the deprecated tools. 24 1.1.1.9.2.3 yamt The next release will drop their code and will stop worrying about 25 1.1.1.9.2.3 yamt backwards compatibility between the ATF libraries and what the old tools 26 1.1.1.9.2.3 yamt may or may not support. 27 1.1.1.9.2.3 yamt 28 1.1.1.9.2.3 yamt If you still require the old tools for some reason, grab a copy of the 29 1.1.1.9.2.3 yamt 'tools' directory now. The code in this directory is standalone and 30 1.1.1.9.2.3 yamt does not depend on any internal details of atf-c++ any longer. 31 1.1.1.9.2.3 yamt 32 1.1.1.9.2.3 yamt * Various fixes and improvements to support running as part of the FreeBSD 33 1.1.1.9.2.3 yamt test suite. 34 1.1.1.9.2.3 yamt 35 1.1.1.9.2.3 yamt * Project hosting moved from Google Code (as a subproject of Kyua) to 36 1.1.1.9.2.3 yamt GitHub (as a first-class project). The main reason for the change is 37 1.1.1.9.2.3 yamt the suppression of binary downloads in Google Code on Jan 15th, 2014. 38 1.1.1.9.2.3 yamt See https://github.com/jmmv/atf/ 39 1.1.1.9.2.3 yamt 40 1.1.1.9.2.3 yamt * Removed builtin help from atf-sh(1) and atf-check(1) for simplicity 41 1.1.1.9.2.3 yamt reasons. In other words, their -h option is gone. 42 1.1.1.9.2.3 yamt 43 1.1.1.9.2.3 yamt * Moved the code of the deprecated tools into a 'tools' directory and 44 1.1.1.9.2.3 yamt completely decoupled their code from the internals of atf-c++. The 45 1.1.1.9.2.3 yamt reason for this is to painlessly allow a third-party to maintain a 46 1.1.1.9.2.3 yamt copy of these tools after we delete them because upcoming changes to 47 1.1.1.9.2.3 yamt atf-c++ would break the stale tools. 48 1.1.1.9.2.3 yamt 49 1.1.1.9.2.3 yamt 50 1.1.1.9.2.3 yamt Changes in version 0.18 51 1.1.1.9.2.3 yamt *********************** 52 1.1.1.9.2.3 yamt 53 1.1.1.9.2.3 yamt Experimental version released on November 16th, 2013. 54 1.1.1.9.2.3 yamt 55 1.1.1.9.2.3 yamt * Issue 45: Added require.memory support in atf-run for FreeBSD. 56 1.1.1.9.2.3 yamt 57 1.1.1.9.2.3 yamt * Fixed an issue with the handling of cin with libc++. 58 1.1.1.9.2.3 yamt 59 1.1.1.9.2.3 yamt * Issue 64: Fixed various mandoc formatting warnings. 60 1.1.1.9.2.3 yamt 61 1.1.1.9.2.3 yamt * NetBSD PR bin/48284: Made atf-check flush its progress message to 62 1.1.1.9.2.3 yamt stdout so that an interrupted test case always shows the last message 63 1.1.1.9.2.3 yamt being executed. 64 1.1.1.9.2.3 yamt 65 1.1.1.9.2.3 yamt * NetBSD PR bin/48285: Fixed atf_check examples in atf-sh-api(3). 66 1.1.1.9.2.3 yamt 67 1.1.1.9.2.3 yamt 68 1.1.1.9.2.3 yamt Changes in version 0.17 69 1.1.1.9.2.3 yamt *********************** 70 1.1.1.9.2.3 yamt 71 1.1.1.9.2.3 yamt Experimental version released on February 14th, 2013. 72 1.1.1.9.2.3 yamt 73 1.1.1.9.2.3 yamt * Added the atf_utils_cat_file, atf_utils_compare_file, 74 1.1.1.9.2.3 yamt atf_utils_copy_file, atf_utils_create_file, atf_utils_file_exists, 75 1.1.1.9.2.3 yamt atf_utils_fork, atf_utils_grep_file, atf_utils_grep_string, 76 1.1.1.9.2.3 yamt atf_utils_readline, atf_utils_redirect and atf_utils_wait utility 77 1.1.1.9.2.3 yamt functions to atf-c-api. Documented the already-public 78 1.1.1.9.2.3 yamt atf_utils_free_charpp function. 79 1.1.1.9.2.3 yamt 80 1.1.1.9.2.3 yamt * Added the cat_file, compare_file, copy_file, create_file, file_exists, 81 1.1.1.9.2.3 yamt fork, grep_collection, grep_file, grep_string, redirect and wait 82 1.1.1.9.2.3 yamt functions to the atf::utils namespace of atf-c++-api. These are 83 1.1.1.9.2.3 yamt wrappers around the same functions added to the atf-c-api library. 84 1.1.1.9.2.3 yamt 85 1.1.1.9.2.3 yamt * Added the ATF_CHECK_MATCH, ATF_CHECK_MATCH_MSG, ATF_REQUIRE_MATCH and 86 1.1.1.9.2.3 yamt ATF_REQUIRE_MATCH_MSG macros to atf-c to simplify the validation of a 87 1.1.1.9.2.3 yamt string against a regular expression. 88 1.1.1.9.2.3 yamt 89 1.1.1.9.2.3 yamt * Miscellaneous fixes for manpage typos and compilation problems with 90 1.1.1.9.2.3 yamt clang. 91 1.1.1.9.2.3 yamt 92 1.1.1.9.2.3 yamt * Added caching of the results of those configure tests that rely on 93 1.1.1.9.2.3 yamt executing a test program. This should help crossbuild systems by 94 1.1.1.9.2.3 yamt providing a mechanism to pre-specify what the results should be. 95 1.1.1.9.2.3 yamt 96 1.1.1.9.2.3 yamt * PR bin/45690: Make atf-report convert any non-printable characters to 97 1.1.1.9.2.3 yamt a plain-text representation (matching their corresponding hexadecimal 98 1.1.1.9.2.3 yamt entities) in XML output files. This is to prevent the output of test 99 1.1.1.9.2.3 yamt cases from breaking xsltproc later. 100 1.1.1.9.2.3 yamt 101 1.1.1.9.2.3 yamt 102 1.1.1.9.2.2 yamt Changes in version 0.16 103 1.1.1.9.2.2 yamt *********************** 104 1.1.1.9.2.2 yamt 105 1.1.1.9.2.2 yamt Experimental version released on July 10th, 2012. 106 1.1.1.9.2.2 yamt 107 1.1.1.9.2.2 yamt * Added a --enable-tools flag to configure to request the build of the 108 1.1.1.9.2.2 yamt deprecated ATF tools, whose build is now disabled by default. In order 109 1.1.1.9.2.2 yamt to continue running tests, you should migrate to Kyua instead of enabling 110 1.1.1.9.2.2 yamt the build of the deprecated tools. The kyua-atf-compat package provides 111 1.1.1.9.2.2 yamt transitional compatibility versions of atf-run and atf-report built on 112 1.1.1.9.2.2 yamt top of Kyua. 113 1.1.1.9.2.2 yamt 114 1.1.1.9.2.2 yamt * Tweaked the ATF_TEST_CASE macro of atf-c++ so that the compiler can 115 1.1.1.9.2.2 yamt detect defined but unused test cases. 116 1.1.1.9.2.2 yamt 117 1.1.1.9.2.2 yamt * PR bin/45859: Fixed some XSLT bugs that resulted in the tc-time and 118 1.1.1.9.2.2 yamt tp-time XML tags leaking into the generated HTML file. Also improved 119 1.1.1.9.2.2 yamt the CSS file slightly to correct alignment and color issues with the 120 1.1.1.9.2.2 yamt timestamps column. 121 1.1.1.9.2.2 yamt 122 1.1.1.9.2.2 yamt * Optimized atf-c++/macros.hpp so that GNU G++ consumes less memory during 123 1.1.1.9.2.2 yamt compilation with GNU G++. 124 1.1.1.9.2.2 yamt 125 1.1.1.9.2.2 yamt * Flipped the default to building shared libraries for atf-c and atf-c++, 126 1.1.1.9.2.2 yamt and started versioning them. As a side-effect, this removes the 127 1.1.1.9.2.2 yamt --enable-unstable-shared flag from configure that appears to not work any 128 1.1.1.9.2.2 yamt more (under NetBSD). Additionally, some distributions require the use of 129 1.1.1.9.2.2 yamt shared libraries for proper dependency tracking (e.g. Fedora), so it is 130 1.1.1.9.2.2 yamt better if we do the right versioning upstream. 131 1.1.1.9.2.2 yamt 132 1.1.1.9.2.2 yamt * Project hosting moved from an adhoc solution (custom web site and 133 1.1.1.9.2.2 yamt Monotone repository) to Google Code (standard wiki and Git). ATF now 134 1.1.1.9.2.2 yamt lives in a subcomponent of the Kyua project. 135 1.1.1.9.2.2 yamt 136 1.1.1.9.2.2 yamt 137 1.1.1.9.2.1 yamt Changes in version 0.15 138 1.1.1.9.2.1 yamt *********************** 139 1.1.1.9.2.1 yamt 140 1.1.1.9.2.1 yamt Experimental version released on January 16th, 2012. 141 1.1.1.9.2.1 yamt 142 1.1.1.9.2.1 yamt * Respect stdin in atf-check. The previous release silenced stdin for any 143 1.1.1.9.2.1 yamt processes spawned by atf, not only test programs, which caused breakage 144 1.1.1.9.2.1 yamt in tests that pipe data through atf-check. 145 1.1.1.9.2.1 yamt 146 1.1.1.9.2.1 yamt * Performance improvements to atf-sh. 147 1.1.1.9.2.1 yamt 148 1.1.1.9.2.1 yamt * Enabled detection of unused parameters and variables in the code and 149 1.1.1.9.2.1 yamt fixed all warnings. 150 1.1.1.9.2.1 yamt 151 1.1.1.9.2.1 yamt * Changed the behavior of "developer mode". Compiler warnings are now 152 1.1.1.9.2.1 yamt enabled unconditionally regardless of whether we are in developer mode or 153 1.1.1.9.2.1 yamt not; developer mode is now only used to perform strict warning checks and 154 1.1.1.9.2.1 yamt to enable assertions. Additionally, developer mode is now only 155 1.1.1.9.2.1 yamt automatically enabled when building from the repository, not for formal 156 1.1.1.9.2.1 yamt releases. 157 1.1.1.9.2.1 yamt 158 1.1.1.9.2.1 yamt * Added new Autoconf M4 macros (ATF_ARG_WITH, ATF_CHECK_C and 159 1.1.1.9.2.1 yamt ATF_CHECK_CXX) to provide a consistent way of defining a --with-arg flag 160 1.1.1.9.2.1 yamt in configure scripts and detecting the presence of any of the ATF 161 1.1.1.9.2.1 yamt bindings. Note that ATF_CHECK_SH was already introduced in 0.14, but it 162 1.1.1.9.2.1 yamt has now been modified to also honor --with-atf if instantiated. 163 1.1.1.9.2.1 yamt 164 1.1.1.9.2.1 yamt * Added timing support to atf-run / atf-report. 165 1.1.1.9.2.1 yamt 166 1.1.1.9.2.1 yamt * Added support for a 'require.memory' property, to specify the minimum 167 1.1.1.9.2.1 yamt amount of physical memory needed by the test case to yield valid results. 168 1.1.1.9.2.1 yamt 169 1.1.1.9.2.1 yamt * PR bin/45690: Force an ISO-8859-1 encoding in the XML files generated by 170 1.1.1.9.2.1 yamt atf-report so that invalid data in the output of test cases does not 171 1.1.1.9.2.1 yamt mangle our report. 172 1.1.1.9.2.1 yamt 173 1.1.1.9.2.1 yamt 174 1.1.1.9 jmmv Changes in version 0.14 175 1.1.1.9 jmmv *********************** 176 1.1.1.9 jmmv 177 1.1.1.9 jmmv Experimental version released on June 14th, 2011. 178 1.1.1.9 jmmv 179 1.1.1.9 jmmv * Added a pkg-config file for atf-sh and an aclocal file to ease the 180 1.1.1.9 jmmv detection of atf-sh from autoconf scripts. 181 1.1.1.9 jmmv 182 1.1.1.9 jmmv * Made the default test case body defined by atf_sh fail. This is to 183 1.1.1.9 jmmv ensure that test cases are properly defined in test programs and helps 184 1.1.1.9 jmmv in catching typos in the names of the body functions. 185 1.1.1.9 jmmv 186 1.1.1.9 jmmv * PR bin/44882: Made atf-run connect the stdin of test cases to /dev/zero. 187 1.1.1.9 jmmv This provides more consistent results with "normal" execution (in 188 1.1.1.9 jmmv particular, when tests are executed detached from a terminal). 189 1.1.1.9 jmmv 190 1.1.1.9 jmmv * Made atf-run hardcode TZ=UTC for test cases. It used to undefine TZ, but 191 1.1.1.9 jmmv that does not take into account that libc determines the current timezone 192 1.1.1.9 jmmv from a configuration file. 193 1.1.1.9 jmmv 194 1.1.1.9 jmmv * All test programs will now print a warning when they are not run through 195 1.1.1.9 jmmv atf-run(1) stating that this is unsupported and may deliver incorrect 196 1.1.1.9 jmmv results. 197 1.1.1.9 jmmv 198 1.1.1.9 jmmv * Added support for the 'require.files' test-case property. This allows 199 1.1.1.9 jmmv test cases to specify installed files that must be present for the test 200 1.1.1.9 jmmv case to run. 201 1.1.1.9 jmmv 202 1.1.1.9 jmmv 203 1.1.1.8 jmmv Changes in version 0.13 204 1.1.1.8 jmmv *********************** 205 1.1.1.8 jmmv 206 1.1.1.8 jmmv Experimental version released on March 31st, 2011. 207 1.1.1.8 jmmv 208 1.1.1.8 jmmv This is the first release after the creation of the Kyua project, a more 209 1.1.1.8 jmmv modular and reliable replacement for ATF. From now on, ATF will change to 210 1.1.1.8 jmmv accomodate the transition to this new codebase, but ATF will still continue 211 1.1.1.8 jmmv to see development in the short/medium term. Check out the project page at 212 1.1.1.8 jmmv http://code.google.com/p/kyua/ for more details. 213 1.1.1.8 jmmv 214 1.1.1.8 jmmv The changes in this release are: 215 1.1.1.8 jmmv 216 1.1.1.8 jmmv * Added support to run the tests with the Kyua runtime engine (kyua-cli), a 217 1.1.1.8 jmmv new package that aims to replace atf-run and atf-report. The ATF tests 218 1.1.1.8 jmmv can be run with the new system by issuing a 'make installcheck-kyua' from 219 1.1.1.8 jmmv the top-level directory of the project (assuming the 'kyua' binary is 220 1.1.1.8 jmmv available during the configuration stage of ATF). 221 1.1.1.8 jmmv 222 1.1.1.8 jmmv * atf-run and atf-report are now in maintenance mode (but *not* deprecated 223 1.1.1.8 jmmv yet!). Kyua already implements a new, much more reliable runtime engine 224 1.1.1.8 jmmv that provides similar features to these tools. That said, it is not 225 1.1.1.8 jmmv complete yet so all development efforts should go towards it. 226 1.1.1.8 jmmv 227 1.1.1.8 jmmv * If GDB is installed, atf-run dumps the stack trace of crashing test 228 1.1.1.8 jmmv programs in an attempt to aid debugging. Contributed by Antti Kantee. 229 1.1.1.8 jmmv 230 1.1.1.8 jmmv * Reverted default timeout change in previous release and reset its value 231 1.1.1.8 jmmv to 5 minutes. This was causing several issues, specially when running 232 1.1.1.8 jmmv the existing NetBSD test suite in qemu. 233 1.1.1.8 jmmv 234 1.1.1.8 jmmv * Fixed the 'match' output checker in atf-check to properly validate the 235 1.1.1.8 jmmv last line of a file even if it does not have a newline. 236 1.1.1.8 jmmv 237 1.1.1.8 jmmv * Added the ATF_REQUIRE_IN and ATF_REQUIRE_NOT_IN macros to atf-c++ to 238 1.1.1.8 jmmv check for the presence (or lack thereof) of an element in a collection. 239 1.1.1.8 jmmv 240 1.1.1.8 jmmv * PR bin/44176: Fixed a race condition in atf-run that would crash atf-run 241 1.1.1.8 jmmv when the cleanup of a test case triggered asynchronous modifications to 242 1.1.1.8 jmmv its work directory (e.g. killing a daemon process that cleans up a pid 243 1.1.1.8 jmmv file in the work directory). 244 1.1.1.8 jmmv 245 1.1.1.8 jmmv * PR bin/44301: Fixed the sample XSLT file to report bogus test programs 246 1.1.1.8 jmmv instead of just listing them as having 0 test cases. 247 1.1.1.8 jmmv 248 1.1.1.8 jmmv 249 1.1.1.7 jmmv Changes in version 0.12 250 1.1.1.7 jmmv *********************** 251 1.1.1.7 jmmv 252 1.1.1.7 jmmv Experimental version released on November 7th, 2010. 253 1.1.1.7 jmmv 254 1.1.1.7 jmmv * Added the ATF_REQUIRE_THROW_RE to atf-c++, which is the same as 255 1.1.1.7 jmmv ATF_REQUIRE_THROW but allows checking for the validity of the exception's 256 1.1.1.7 jmmv error message by means of a regular expression. 257 1.1.1.7 jmmv 258 1.1.1.7 jmmv * Added the ATF_REQUIRE_MATCH to atf-c++, which allows checking for a 259 1.1.1.7 jmmv regular expression match in a string. 260 1.1.1.7 jmmv 261 1.1.1.7 jmmv * Changed the default timeout for test cases from 5 minutes to 30 seconds. 262 1.1.1.7 jmmv 30 seconds is long enough for virtually all tests to complete, and 5 263 1.1.1.7 jmmv minutes is a way too long pause in a test suite where a single test case 264 1.1.1.7 jmmv stalls. 265 1.1.1.7 jmmv 266 1.1.1.7 jmmv * Deprecated the use.fs property. While this seemed like a good idea in 267 1.1.1.7 jmmv the first place to impose more control on what test cases can do, it 268 1.1.1.7 jmmv turns out to be bad. First, use.fs=false prevents bogus test cases 269 1.1.1.7 jmmv from dumping core so after-the-fact debugging is harder. Second, 270 1.1.1.7 jmmv supporting use.fs adds a lot of unnecessary complexity. atf-run will 271 1.1.1.7 jmmv now ignore any value provided to use.fs and will allow test cases to 272 1.1.1.7 jmmv freely access the file system if they wish to. 273 1.1.1.7 jmmv 274 1.1.1.7 jmmv * Added the atf_tc_get_config_var_as_{bool,long}{,_wd} functions to the atf-c 275 1.1.1.7 jmmv library. The 'text' module became private in 0.11 but was being used 276 1.1.1.7 jmmv externally to simplify the parsing of configuration variables. 277 1.1.1.7 jmmv 278 1.1.1.7 jmmv * Made atf-run recognize the 'unprivileged-user' configuration variable 279 1.1.1.7 jmmv and automatically drop root privileges when a test case sets 280 1.1.1.7 jmmv require.user=unprivileged. Note that this is, by no means, done for 281 1.1.1.7 jmmv security purposes; this is just for user convenience; tests should, in 282 1.1.1.7 jmmv general, not be blindly run as root in the first place. 283 1.1.1.7 jmmv 284 1.1.1.7 jmmv 285 1.1.1.6 jmmv Changes in version 0.11 286 1.1.1.6 jmmv *********************** 287 1.1.1.6 jmmv 288 1.1.1.6 jmmv Experimental version released on October 20th, 2010. 289 1.1.1.6 jmmv 290 1.1.1.6 jmmv * The ATF_CHECK* macros in atf-c++ were renamed to ATF_REQUIRE* to match 291 1.1.1.6 jmmv their counterparts in atf-c. 292 1.1.1.6 jmmv 293 1.1.1.6 jmmv * Clearly separated the modules in atf-c that are supposed to be public 294 1.1.1.6 jmmv from those that are implementation details. The header files for the 295 1.1.1.6 jmmv internal modules are not installed any more. 296 1.1.1.6 jmmv 297 1.1.1.6 jmmv * Made the atf-check tool private. It is only required by atf-sh and being 298 1.1.1.6 jmmv public has the danger of causing confusion. Also, making it private 299 1.1.1.6 jmmv simplifies the public API of atf. 300 1.1.1.6 jmmv 301 1.1.1.6 jmmv * Changed atf-sh to enable per-command error checking (set -e) by default. 302 1.1.1.6 jmmv This catches many cases in which a test case is broken but it is not 303 1.1.1.6 jmmv reported as such because execution continues. 304 1.1.1.6 jmmv 305 1.1.1.6 jmmv * Fixed the XSTL and CSS stylesheets to support expected failures. 306 1.1.1.6 jmmv 307 1.1.1.6 jmmv 308 1.1.1.5 jmmv Changes in version 0.10 309 1.1.1.5 jmmv *********************** 310 1.1 jmmv 311 1.1.1.5 jmmv Experimental version released on July 2nd, 2010. 312 1.1 jmmv 313 1.1.1.5 jmmv Miscellaneous features 314 1.1 jmmv 315 1.1.1.5 jmmv * Added expected failures support to test cases and atf-run. These 316 1.1.1.5 jmmv include, for example, expected clean exits, expected reception of fatal 317 1.1.1.5 jmmv signals, expected timeouts and expected errors in condition checks. 318 1.1.1.5 jmmv These statuses can be used to denote test cases that are known to fail 319 1.1.1.5 jmmv due to a bug in the code they are testing. atf-report reports these 320 1.1.1.5 jmmv tests separately but they do not count towards the failed test cases 321 1.1.1.5 jmmv amount. 322 1.1 jmmv 323 1.1.1.5 jmmv * Added the ATF_CHECK_ERRNO and ATF_REQUIRE_ERRNO to the C library to 324 1.1.1.5 jmmv allow easy checking of call failures that update errno. 325 1.1 jmmv 326 1.1.1.5 jmmv * Added the has.cleanup meta-data property to test caes that specifies 327 1.1.1.5 jmmv whether the test case has a cleanup routine or not; its value is 328 1.1.1.5 jmmv automatically set. This property is read by atf-run to know if it has to 329 1.1.1.5 jmmv run the cleanup routine; skipping this run for every test case 330 1.1.1.5 jmmv significantly speeds up the run time of test suites. 331 1.1 jmmv 332 1.1.1.5 jmmv * Reversed the order of the ATF_CHECK_THROW macro in the C++ binding to 333 1.1.1.5 jmmv take the expected exception as the first argument and the statement to 334 1.1.1.5 jmmv execute as the second argument. 335 1.1 jmmv 336 1.1.1.5 jmmv Changes in atf-check 337 1.1.1.3 jmmv 338 1.1.1.5 jmmv * Changed atf-check to support negating the status and output checks by 339 1.1.1.5 jmmv prefixing them with not- and added support to specify multiple checkers 340 1.1.1.5 jmmv for stdout and stderr, not only one. 341 1.1.1.4 jmmv 342 1.1.1.5 jmmv * Added the match output checker to atf-check to look for regular 343 1.1.1.5 jmmv expressions in the stdout and stderr of commands. 344 1.1.1.4 jmmv 345 1.1.1.5 jmmv * Modified the exit checks in atf-check to support checking for the 346 1.1.1.5 jmmv reception of signals. 347 1.1.1.4 jmmv 348 1.1.1.5 jmmv Code simplifications and cleanups 349 1.1.1.4 jmmv 350 1.1.1.5 jmmv * Removed usage messages from test programs to simplify the 351 1.1.1.5 jmmv implementation of every binding by a significant amount. They just now 352 1.1.1.5 jmmv refer the user to the appropriate manual page and do not attempt to wrap 353 1.1.1.5 jmmv lines on terminal boundaries. Test programs are not supposed to be run 354 1.1.1.5 jmmv by users directly so this minor interface regression is not important. 355 1.1.1.4 jmmv 356 1.1.1.5 jmmv * Removed the atf-format internal utility, which is unused after the 357 1.1.1.5 jmmv change documented above. 358 1.1.1.4 jmmv 359 1.1.1.5 jmmv * Removed the atf-cleanup internal utility. It has been unused since the 360 1.1.1.5 jmmv test case isolation was moved to atf-run in 0.8 361 1.1.1.4 jmmv 362 1.1.1.5 jmmv * Splitted the Makefile.am into smaller files for easier maintenance and 363 1.1.1.5 jmmv dropped the use of M4. Only affects users building from the repository 364 1.1.1.5 jmmv sources. 365 1.1.1.4 jmmv 366 1.1.1.5 jmmv * Intermixed tests with the source files in the source tree to provide 367 1.1.1.5 jmmv them more visibility and easier access. The tests directory is gone from 368 1.1.1.5 jmmv the source tree and tests are now suffixed by _test, not prefixed by t_. 369 1.1.1.3 jmmv 370 1.1.1.5 jmmv * Simplifications to the atf-c library: removed the io, tcr and ui 371 1.1.1.5 jmmv modules as they had become unnecessary after all simplifications 372 1.1.1.5 jmmv introduced since the 0.8 release. 373 1.1.1.3 jmmv 374 1.1.1.5 jmmv * Removed the application/X-atf-tcr format introduced in 0.8 release. 375 1.1.1.5 jmmv Tests now print a much simplified format that is easy to parse and nicer 376 1.1.1.5 jmmv to read by end users. As a side effect, the default for test cases is 377 1.1.1.5 jmmv now to print their results to stdout unless otherwise stated by providing 378 1.1.1.5 jmmv the -r flag. 379 1.1.1.3 jmmv 380 1.1.1.5 jmmv * Removed XML distribution documents and replaced them with plain-text 381 1.1.1.5 jmmv documents. They provided little value and introduced a lot of complexity 382 1.1.1.5 jmmv to the build system. 383 1.1.1.3 jmmv 384 1.1.1.5 jmmv * Simplified the output of atf-version by not attempting to print a 385 1.1.1.5 jmmv revision number when building form a distfile. Makes the build system 386 1.1.1.5 jmmv easier to maintain. 387 1.1.1.3 jmmv 388 1.1.1.3 jmmv 389 1.1.1.5 jmmv Changes in version 0.9 390 1.1.1.5 jmmv ********************** 391 1.1.1.3 jmmv 392 1.1.1.5 jmmv Experimental version released on June 3rd, 2010. 393 1.1.1.3 jmmv 394 1.1.1.5 jmmv * Added atf-sh, an interpreter to process test programs written using 395 1.1.1.5 jmmv the shell API. This is not really a shell interpreter by itself though: 396 1.1.1.5 jmmv it is just a wrapper around the system shell that eases the loading of 397 1.1.1.5 jmmv the necessary ATF libraries. 398 1.1.1.3 jmmv 399 1.1.1.5 jmmv * Removed atf-compile in favour of atf-sh. 400 1.1.1.3 jmmv 401 1.1.1.5 jmmv * Added the use.fs metadata property to test case, which is used to 402 1.1.1.5 jmmv specify which test cases require file system access. This is to 403 1.1.1.5 jmmv highlight dependencies on external resources more clearly and to speed up 404 1.1.1.5 jmmv the execution of test suites by skipping the creation of many unnecessary 405 1.1.1.5 jmmv work directories. 406 1.1.1.3 jmmv 407 1.1.1.5 jmmv * Fixed test programs to get a sane default value for their source 408 1.1.1.5 jmmv directory. This means that it should not be necessary any more to pass 409 1.1.1.5 jmmv -s when running test programs that do not live in the current directory. 410 1.1 jmmv 411 1.1.1.5 jmmv * Defining test case headers became optional. This is trivial to achieve 412 1.1.1.5 jmmv in shell-based tests but a bit ugly in C and C++. In C, use the new 413 1.1.1.5 jmmv ATF_TC_WITHOUT_HEAD macro to define the test case, and in C++ use 414 1.1.1.5 jmmv ATF_TEST_CASE_WITHOUT_HEAD. 415 1.1 jmmv 416 1.1 jmmv 417 1.1.1.5 jmmv Changes in version 0.8 418 1.1.1.5 jmmv ********************** 419 1.1 jmmv 420 1.1.1.5 jmmv Experimental version released on May 7th, 2010. 421 1.1 jmmv 422 1.1.1.5 jmmv * Test programs no longer run several test cases in a row. The execution 423 1.1.1.5 jmmv of a test program now requires a test case name, and that single test 424 1.1.1.5 jmmv case is executed. To execute several test cases, use the atf-run utility 425 1.1.1.5 jmmv as usual. 426 1.1 jmmv 427 1.1.1.5 jmmv * Test programs no longer fork a subprocess to isolate the execution of 428 1.1.1.5 jmmv test cases. They run the test case code in-process, and a crash of the 429 1.1.1.5 jmmv test case will result in a crash of the test program. This is to ease 430 1.1.1.5 jmmv debugging of faulty test cases. 431 1.1 jmmv 432 1.1.1.5 jmmv * Test programs no longer isolate their test cases. This means that they 433 1.1.1.5 jmmv will not create temporary directories nor sanitize the environment any 434 1.1.1.5 jmmv more. Yes: running a test case that depends on system state by hand will 435 1.1.1.5 jmmv most likely yield different results depending on where (machine, 436 1.1.1.5 jmmv directory, user environment, etc.) it is run. Isolation has been moved 437 1.1.1.5 jmmv to atf-run. 438 1.1 jmmv 439 1.1.1.5 jmmv * Test programs no longer print a cryptic format (application/X-atf-tcs) 440 1.1.1.5 jmmv on a special file channel. They can now print whatever they want on the 441 1.1.1.5 jmmv screen. Because test programs can now only run one test case every time, 442 1.1.1.5 jmmv providing controlled output is not necessary any more. 443 1.1 jmmv 444 1.1.1.5 jmmv * Test programs no longer write their status into a special file 445 1.1.1.5 jmmv descriptor. Instead, they create a file with the results, which is later 446 1.1.1.5 jmmv parsed by atf-run. This changes the semantics of the -r flag. 447 1.1 jmmv 448 1.1.1.5 jmmv * atf-run has been adjusted to perform the test case isolation. As a 449 1.1.1.5 jmmv result, there is now a single canonical place that implements the 450 1.1.1.5 jmmv isolation of test caes. In previous releases, the three language 451 1.1.1.5 jmmv bindings (C, C++ and shell) had to be kept in sync with each other (read: 452 1.1.1.5 jmmv not a nice thing to do at all). As a side effect of this change, writing 453 1.1.1.5 jmmv bindings for other languages will be much, much easier from now on. 454 1.1 jmmv 455 1.1.1.5 jmmv * atf-run forks test programs on a test case basis, instead of on a test 456 1.1.1.5 jmmv program basis as it did before. This is to provide the test case 457 1.1.1.5 jmmv isolation that was before implemented by the test programs themselves. 458 1.1 jmmv 459 1.1.1.5 jmmv * Removed the atf-exec tool. This was used to implement test case 460 1.1.1.5 jmmv isolation in atf-sh, but it is now unnecessary. 461 1.1 jmmv 462 1.1.1.5 jmmv * It is now optional to define the descr meta-data property. It has been 463 1.1.1.5 jmmv proven to be mostly useless, because test cases often carry a descriptive 464 1.1.1.5 jmmv name of their own. 465 1.1 jmmv 466 1.1 jmmv 467 1.1.1.5 jmmv Changes in version 0.7 468 1.1.1.5 jmmv ********************** 469 1.1 jmmv 470 1.1.1.5 jmmv Experimental version released on December 22nd, 2009. 471 1.1 jmmv 472 1.1.1.5 jmmv * Added build-time checks to atf-c and atf-c++. A binding for atf-sh 473 1.1.1.5 jmmv will come later. 474 1.1 jmmv 475 1.1.1.5 jmmv * Migrated all build-time checks for header files to proper ATF tests. 476 1.1.1.5 jmmv This demonstrates the use of the new feature described above. 477 1.1 jmmv 478 1.1.1.5 jmmv * Added an internal API for child process management. 479 1.1 jmmv 480 1.1.1.5 jmmv * Converted all plain-text distribution documents to a Docbook canonical 481 1.1.1.5 jmmv version, and include pre-generated plain text and HTML copies in the 482 1.1.1.5 jmmv distribution file. 483 1.1 jmmv 484 1.1.1.5 jmmv * Simplified the contents of the Makefile.am by regenerating it from a 485 1.1.1.5 jmmv canonical Makefile.am.m4 source. As a side-effect, some dependency 486 1.1.1.5 jmmv specifications were fixed. 487 1.1 jmmv 488 1.1.1.5 jmmv * Migrated all checks from the check target to installcheck, as these 489 1.1.1.5 jmmv require ATF to be installed. 490 1.1 jmmv 491 1.1.1.5 jmmv * Fixed sign comparison mismatches triggered by the now-enabled 492 1.1.1.5 jmmv -Wsign-compare. 493 1.1 jmmv 494 1.1.1.5 jmmv * Fixed many memory and object leaks. 495 1.1 jmmv 496 1.1 jmmv 497 1.1.1.5 jmmv Changes in version 0.6 498 1.1.1.5 jmmv ********************** 499 1.1 jmmv 500 1.1.1.5 jmmv Experimental version released on January 18th, 2009. 501 1.1 jmmv 502 1.1.1.5 jmmv * Make atf-exec be able to kill its child process after a certain period 503 1.1.1.5 jmmv of time; this is controlled through the new -t option. 504 1.1 jmmv 505 1.1.1.5 jmmv * Change atf-sh to use atf-exec's -t option to control the test case's 506 1.1.1.5 jmmv timeouts, instead of doing it internally. Same behavior as before, but 507 1.1.1.5 jmmv noticeably faster. 508 1.1 jmmv 509 1.1.1.5 jmmv * atf-exec's -g option and atf-killpg are gone due to the previous 510 1.1.1.5 jmmv change. 511 1.1 jmmv 512 1.1.1.5 jmmv * Added the atf-check(1) tool, a program that executes a given command 513 1.1.1.5 jmmv and checks its exit code against a known value and allows the management 514 1.1.1.5 jmmv of stdout and stderr in multiple ways. This replaces the previous 515 1.1.1.5 jmmv atf_check function in the atf-sh library and exposes this functionality 516 1.1.1.5 jmmv to both atf-c and atf-c++. 517 1.1 jmmv 518 1.1.1.5 jmmv * Added the ATF_REQUIRE family of macros to the C interface. These help 519 1.1.1.5 jmmv in checking for fatal test conditions. The old ATF_CHECK macros now 520 1.1.1.5 jmmv perform non-fatal checks only. I.e. by using ATF_CHECK, the test case 521 1.1.1.5 jmmv can now continue its execution and the failures will not be reported 522 1.1.1.5 jmmv until the end of the whole run. 523 1.1 jmmv 524 1.1.1.5 jmmv * Extended the amount of ATF_CHECK_* C macros with new ones to provide 525 1.1.1.5 jmmv more features to the developer. These also have their corresponding 526 1.1.1.5 jmmv counterparts in the ATF_REQUIRE_* family. The new macros (listing the 527 1.1.1.5 jmmv suffixes only) are: _EQ (replaces _EQUAL), _EQ_MSG, _STREQ and 528 1.1.1.5 jmmv _STREQ_MSG. 529 1.1 jmmv 530 1.1 jmmv 531 1.1.1.5 jmmv Changes in version 0.5 532 1.1.1.5 jmmv ********************** 533 1.1 jmmv 534 1.1.1.5 jmmv Experimental version released on May 1st, 2008. 535 1.1 jmmv 536 1.1.1.5 jmmv * Clauses 3 and 4 of the BSD license used by the project were dropped. 537 1.1.1.5 jmmv All the code is now under a 2-clause BSD license compatible with the GNU 538 1.1.1.5 jmmv General Public License (GPL). 539 1.1 jmmv 540 1.1.1.5 jmmv * Added a C-only binding so that binary test programs do not need to be 541 1.1.1.5 jmmv tied to C++ at all. This binding is now known as the atf-c library. 542 1.1 jmmv 543 1.1.1.5 jmmv * Renamed the C++ binding to atf-c++ for consistency with the new atf-c. 544 1.1 jmmv 545 1.1.1.5 jmmv * Renamed the POSIX shell binding to atf-sh for consistency with the new 546 1.1.1.5 jmmv atf-c and atf-c++. 547 1.1 jmmv 548 1.1.1.5 jmmv * Added a -w flag to test programs through which it is possible to 549 1.1.1.5 jmmv specify the work directory to be used. This was possible in prior 550 1.1.1.5 jmmv releases by defining the workdir configuration variable (-v workdir=...), 551 1.1.1.5 jmmv but was a conceptually incorrect mechanism. 552 1.1 jmmv 553 1.1.1.5 jmmv * Test programs now preserve the execution order of test cases when they 554 1.1.1.5 jmmv are given in the command line. Even those mentioned more than once are 555 1.1.1.5 jmmv executed multiple times to comply with the user's requests. 556 1.1 jmmv 557 1.1 jmmv 558 1.1.1.5 jmmv Changes in version 0.4 559 1.1.1.5 jmmv ********************** 560 1.1 jmmv 561 1.1.1.5 jmmv Experimental version released on February 4th, 2008. 562 1.1 jmmv 563 1.1.1.5 jmmv * Added two new manual pages, atf-c++-api and atf-sh-api, describing the 564 1.1.1.5 jmmv C++ and POSIX shell interfaces used to write test programs. 565 1.1 jmmv 566 1.1.1.5 jmmv * Added a pkg-config file, useful to get the flags to build against the 567 1.1.1.5 jmmv C++ library or to easily detect the presence of ATF. 568 1.1 jmmv 569 1.1.1.5 jmmv * Added a way for test cases to require a specific architecture and/or 570 1.1.1.5 jmmv machine type through the new 'require.arch' and 'require.machine' 571 1.1.1.5 jmmv meta-data properties, respectively. 572 1.1 jmmv 573 1.1.1.5 jmmv * Added the 'timeout' property to test cases, useful to set an 574 1.1.1.5 jmmv upper-bound limit for the test's run time and thus prevent global test 575 1.1.1.5 jmmv program stalls due to the test case's misbehavior. 576 1.1 jmmv 577 1.1.1.5 jmmv * Added the atf-exec(1) internal utility, used to execute a command 578 1.1.1.5 jmmv after changing the process group it belongs to. 579 1.1.1.2 jmmv 580 1.1.1.5 jmmv * Added the atf-killpg(1) internal utility, used to kill process groups. 581 1.1.1.2 jmmv 582 1.1.1.5 jmmv * Multiple portability fixes. Of special interest, full support for 583 1.1.1.5 jmmv SunOS (Solaris Express Developer Edition 2007/09) using the Sun Studio 12 584 1.1.1.5 jmmv C++ compiler. 585 1.1.1.2 jmmv 586 1.1.1.5 jmmv * Fixed a serious bug that prevented atf-run(1) from working at all 587 1.1.1.5 jmmv under Fedora 8 x86_64. Due to the nature of the bug, other platforms 588 1.1.1.5 jmmv were likely affected too. 589 1.1.1.2 jmmv 590 1.1.1.2 jmmv 591 1.1.1.5 jmmv Changes in version 0.3 592 1.1.1.5 jmmv ********************** 593 1.1.1.2 jmmv 594 1.1.1.5 jmmv Experimental version released on November 11th, 2007. 595 1.1.1.2 jmmv 596 1.1.1.5 jmmv * Added XML output support to atf-report. This is accompanied by a DTD 597 1.1.1.5 jmmv for the format's structure and sample XSLT/CSS files to post-process this 598 1.1.1.5 jmmv output and convert it to a plain HTML report. 599 1.1.1.5 jmmv 600 1.1.1.5 jmmv * Changed atf-run to add system information to the report it generates. 601 1.1.1.5 jmmv This is currently used by atf-report's XML output only, and is later 602 1.1.1.5 jmmv printed in the HTML reports in a nice and useful summary table. The user 603 1.1.1.5 jmmv and system administrator are allowed to tune this feature by means of 604 1.1.1.5 jmmv hooks. 605 1.1.1.5 jmmv 606 1.1.1.5 jmmv * Removed the test cases' 'isolated' property. This was intended to 607 1.1.1.5 jmmv avoid touching the file system at all when running the related test case, 608 1.1.1.5 jmmv but this has not been true for a long while: some control files are 609 1.1.1.5 jmmv unconditionally required for several purposes, and we cannot easily get 610 1.1.1.5 jmmv rid of them. This way we remove several critical and delicate pieces of 611 1.1.1.5 jmmv code. 612 1.1.1.5 jmmv 613 1.1.1.5 jmmv * Improved atf-report's CSV output format to include information about 614 1.1.1.5 jmmv test programs too. 615 1.1.1.5 jmmv 616 1.1.1.5 jmmv * Fixed the tests that used atf-compile to not require this tool as a 617 1.1.1.5 jmmv helper. Avoids systems without build-time utilities to skip many tests 618 1.1.1.5 jmmv that could otherwise be run. (E.g. NetBSD without the comp.tgz set 619 1.1.1.5 jmmv installed.) 620 1.1.1.5 jmmv 621 1.1.1.5 jmmv * Many general cleanups: Fixed many pieces of code marked as ugly and/or 622 1.1.1.5 jmmv incomplete. 623 1.1.1.5 jmmv 624 1.1.1.5 jmmv 625 1.1.1.5 jmmv Changes in version 0.2 626 1.1.1.5 jmmv ********************** 627 1.1.1.5 jmmv 628 1.1.1.5 jmmv Experimental version released on September 20th, 2007. 629 1.1.1.5 jmmv 630 1.1.1.5 jmmv * Test cases now get a known umask on entry. 631 1.1.1.5 jmmv 632 1.1.1.5 jmmv * atf-run now detects many unexpected failures caused by test programs and 633 1.1.1.5 jmmv reports them as bogus tests. atf-report is able to handle these new 634 1.1.1.5 jmmv errors and nicely reports them to the user. 635 1.1.1.5 jmmv 636 1.1.1.5 jmmv * All the data formats read and written by the tools have been 637 1.1.1.5 jmmv documented and cleaned up. These include those grammars that define how 638 1.1.1.5 jmmv the different components communicate with each other as well as the 639 1.1.1.5 jmmv format of files written by the developers and users: the Atffiles and the 640 1.1.1.5 jmmv configuration files. 641 1.1.1.5 jmmv 642 1.1.1.5 jmmv * Added the atf-version tool, a utility that displays information about 643 1.1.1.5 jmmv the currently installed version of ATF. 644 1.1.1.5 jmmv 645 1.1.1.5 jmmv * Test cases can now define an optional cleanup routine to undo their 646 1.1.1.5 jmmv actions regardless of their exit status. 647 1.1.1.5 jmmv 648 1.1.1.5 jmmv * atf-report now summarizes the list of failed (bogus) test programs 649 1.1.1.5 jmmv when using the ticker output format. 650 1.1.1.5 jmmv 651 1.1.1.5 jmmv * Test programs now capture some termination signals and clean up any 652 1.1.1.5 jmmv temporary files before exiting the program. 653 1.1.1.5 jmmv 654 1.1.1.5 jmmv * Multiple bug fixes and improvements all around. 655 1.1.1.5 jmmv 656 1.1.1.5 jmmv 657 1.1.1.5 jmmv Changes in version 0.1 658 1.1.1.5 jmmv ********************** 659 1.1.1.5 jmmv 660 1.1.1.5 jmmv Experimental version released on August 20th, 2007. 661 1.1.1.5 jmmv 662 1.1.1.5 jmmv * First public version. This was released coinciding with the end of the 663 1.1.1.5 jmmv Google Summer of Code 2007 program. 664 1.1.1.5 jmmv 665 1.1.1.5 jmmv 666 1.1.1.5 jmmv =========================================================================== 667 1.1.1.5 jmmv vim: filetype=text:textwidth=75:expandtab:shiftwidth=2:softtabstop=2 668
Indexes created Fri Sep 11 00:24:00 UTC 2026