xref: /src/external/bsd/atf/dist/NEWS

                     Automated Testing Framework (atf)
                        The NetBSD Foundation, Inc.
---------------------------------------------------------------------------


Version 0.6
===========

Release date: January 18th, 2009
Status:       Experimental

* Make atf-exec be able to kill its child process after a certain period of
  time; this is controlled through the new -t option.

* Change atf-sh to use atf-exec's -t option to control the test case's
  timeouts, instead of doing it internally.  Same behavior as before, but
  noticeably faster.

* atf-exec's -g option and atf-killpg are gone due to the previous change.

* Added the atf-check(1) tool, a program that executes a given command and
  checks its exit code against a known value and allows the management of
  stdout and stderr in multiple ways.  This replaces the previous atf_check
  function in the atf-sh library and exposes this functionality to both
  atf-c and atf-c++.

* Added the ATF_REQUIRE family of macros to the C interface.  These help
  in checking for fatal test conditions.  The old ATF_CHECK macros now
  perform non-fatal checks only.  I.e. by using ATF_CHECK, the test case
  can now continue its execution and the failures will not be reported
  until the end of the whole run.

* Extended the amount of ATF_CHECK_* C macros with new ones to provide more
  features to the developer.  These also have their corresponding
  counterparts in the ATF_REQUIRE_* family.  The new macros (listing the
  suffixes only) are: _EQ (replaces _EQUAL), _EQ_MSG, _STREQ and
  _STREQ_MSG.


Version 0.5
===========

Release date: May 1st, 2008
Status:       Experimental

* Clauses 3 and 4 of the BSD license used by the project were dropped.
  All the code is now under a 2-clause BSD license compatible with the
  GNU General Public License (GPL).

* Added a C-only binding so that binary test programs do not need to be
  tied to C++ at all.  This binding is now known as the atf-c library.

* Renamed the C++ binding to atf-c++ for consistency with the new atf-c.

* Renamed the POSIX shell binding to atf-sh for consistency with the new
  atf-c and atf-c++.

* Added a -w flag to test programs through which it is possible to specify
  the work directory to be used.  This was possible in prior releases by
  defining the workdir configuration variable (-v workdir=...), but was a
  conceptually incorrect mechanism.

* Test programs now preserve the execution order of test cases when they
  are given in the command line.  Even those mentioned more than once are
  executed multiple times to comply with the user's requests.


Version 0.4
===========

Release date: February 4th, 2008
Status:       Experimental

Changes:

* Added two new manual pages, atf-c++-api and atf-sh-api, describing the
  C++ and POSIX shell interfaces used to write test programs.

* Added a pkg-config file, useful to get the flags to build against the
  C++ library or to easily detect the presence of ATF.

* Added a way for test cases to require a specific architecture and/or
  machine type through the new 'require.arch' and 'require.machine'
  meta-data properties, respectively.

* Added the 'timeout' property to test cases, useful to set an upper-bound
  limit for the test's run time and thus prevent global test program stalls
  due to the test case's misbehavior.

* Added the atf-exec(1) internal utility, used to execute a command after
  changing the process group it belongs to.

* Added the atf-killpg(1) internal utility, used to kill process groups.

* Multiple portability fixes.  Of special interest, full support for SunOS
  (Solaris Express Developer Edition 2007/09) using the Sun Studio 12 C++
  compiler.

* Fixed a serious bug that prevented atf-run(1) from working at all under
  Fedora 8 x86_64.  Due to the nature of the bug, other platforms were
  likely affected too.


Version 0.3
===========

Release date: November 11th, 2007
Status:       Experimental

Changes:

* Added XML output support to atf-report.  This is accompanied by a DTD for
  the format's structure and sample XSLT/CSS files to post-process this
  output and convert it to a plain HTML report.

* Changed atf-run to add system information to the report it generates.
  This is currently used by atf-report's XML output only, and is later
  printed in the HTML reports in a nice and useful summary table.  The user
  and system administrator are allowed to tune this feature by means of
  hooks.

* Removed the test cases' 'isolated' property.  This was intended to avoid
  touching the file system at all when running the related test case, but
  this has not been true for a long while: some control files are
  unconditionally required for several purposes, and we cannot easily get
  rid of them.  This way we remove several critical and delicate pieces of
  code.

* Improved atf-report's CSV output format to include information about
  test programs too.

* Fixed the tests that used atf-compile to not require this tool as a
  helper.  Avoids systems without build-time utilities to skip many tests
  that could otherwise be run.  (E.g. NetBSD without the comp.tgz set
  installed.)

* Many general cleanups: Fixed many pieces of code marked as ugly and/or
  incomplete.


Version 0.2
===========

Release date: September 20th, 2007
Status:       Experimental

Changes:

* Test cases now get a known umask on entry.

* atf-run now detects many unexpected failures caused by test programs
  and reports them as bogus tests.  atf-report is able to handle these
  new errors and nicely reports them to the user.

* All the data formats read and written by the tools have been documented
  and cleaned up.  These include those grammars that define how the
  different components communicate with each other as well as the format
  of files written by the developers and users: the Atffiles and the
  configuration files.

* Added the atf-version tool, a utility that displays information about the
  currently installed version of ATF.

* Test cases can now define an optional cleanup routine to undo their
  actions regardless of their exit status.

* atf-report now summarizes the list of failed (bogus) test programs when
  using the ticker output format.

* Test programs now capture some termination signals and clean up any
  temporary files before exiting the program.

* Multiple bug fixes and improvements all around.


Version 0.1
===========

Release date: August 20th, 2007
Status:       Experimental

Changes:

* First public version.  This was released coinciding with the end of the
  Google Summer of Code 2007 program.


---------------------------------------------------------------------------
                              End of document