Universal test output production and consumption facility

testbild

testbild - Universal test output production and consumption facility
====================================================================
testbild is a Common Lisp library designed to provide a common interface for
[Unit Testing] [test] output. Currently it supports [TAP] [tap] (versions 12,
13) and [xUnit] [xunit] style output.

Support for TAP consumption is planned.

testbild is completely written in Common Lisp and licensed under the
[GPLv3+] [gpl]. Please see the file COPYING in the top-level directory of the
distribution tarball or see the link above if you didn't receive a copy along
with this file.


Installation
------------
Please see file INSTALL.


Usage
-----
If you're using a testing framework, maybe it already offers support for
testbild. If not, go and push its authors to check it out or maybe even provide
them with patches that integrate the framework with testbild yourself!

In case you're one of said authors or doing the testing all by yourself, read
on:

Generally, testbild is comprised by a set of CLOS classes, `producer` being the
root class. It defines the minimal interface which is mostly influenced by TAP,
the output style known for offering most capabilities, but implements void
default methods for all generics save from `emit-result` which is the bare
minimum to implement for a test output class. `producer` also (re)sets and
internal test counter upon (re)initialization available via the accessor
`tests-run`, thus no additional effort has to be put into test producers relying
on test counters.

Output producer implementations should only differ in instance initialization
arguments and keywords allowed as values for `directive`, one of `emit-result`'s
`&key` arguments, hence being the only things differing between subclasses to be
taken care of by its user.

Integrating low-level test code with testbild should be done as follows:

* Generate a fresh test producer instance that is tied to the execution of a
  test suite, file etc., e.g. by creating a closure or using a special variable
* Couple the start of test code execution with `init-test`
* Same for termination and `finalize-test`
* To comply with TAP, have the user define a "test plan" no matter which output
  they choose and execute `emit-plan` right after `init-test` or right before
  `finalize-test`
* Use and/or provide `emit-comment` and `emit-bailout` as desired

A typical use case might be as follows:

    CL-USER> (let ((producer (make-instance 'tap-producer)))
               (init-test producer)
               (emit-plan producer :plan-argument 3) ; simple plan, three tests planned
               (emit-result producer :success (= (+ 1 2) 3) :description "test whether 1+2 equals 3")
               (emit-comment producer "Math is fun!")
               (emit-result producer :success (eql nil 42) :description "42 is the answer!" :directive :todo :reason "But we're still in hunt of the question..")
               (emit-result producer :success t :directive :skip :reason "Skipping stuff is also supported")
               (finalize-test producer))
    TAP version 13
    1..3
    ok 1 - test whether 1+2 equals 3
    # Math is fun!
    not ok 2 - 42 is the answer! # TODO But we're still in hunt of the question..
    ok 3 # SKIP Skipping stuff is also supported

If _x-unit-producer_ was used instead, this would have been the output:

    .IS
on STDOUT and

    Math is fun!
on STDERR as xUnit has no native support for comments.


Missing Stuff
-------------
* TAP
  * Support for [YAMLish] [yamlish] diagnostics
  * Consumer part
* Other formats
  * Story / BDD
  * TestDox

History
-------
While there are many nice testing framework for Common Lisp and writing your own
test code is especially easy with the most powerful language on earth, most if
not all of said frameworks use an inferior report style usually found in and
associated with the xUnit family of unit testing frameworks, even though there
is a formally specified, clearly superior alternative available: TAP, Test
Anything Protocol.

In lack of TAP support _Testbild_, which is the German word for (TV) test
pattern but also interpretable as (overall) "test appearance", was written to
help both test and test framework authors exploit the features of TAP without
losing support for different reporting styles.


Links and References
--------------------
Homepage: <http://www.cliki.net/testbild>  
Hacking:  <http://github.com/e-user/testbild>

[gpl]: http://www.gnu.org/licenses/gpl-3.0-standalone.html
[test]: http://en.wikipedia.org/wiki/Unit_testing
[tap]: http://testanything.org/
[xunit]: http://en.wikipedia.org/wiki/XUnit
[yamlish]: http://testanything.org/wiki/index.php?title=YAMLish&oldid=1930



Copyright
---------
Copyright (C) 2010  Alexander Kahl <e-user@fsfe.org>  
This file is part of testbild.  
testbild is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.

testbild is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.


testbild

Upstream URL

Author

License

testbild

Version:

License:

Authors:

Dependencies:

testbild-test

Version:

License:

Authors:

Dependencies:

Dependencies (5)

Dependents (0)