fiveam

API Reference

fiveam

A simple regression testing framework

IT.BESE.FIVEAM

  • Variable *TEST-DRIBBLE*
    t
  • Macro IS (test &rest reason-args)
    The DWIM checking operator. If TEST returns a true value a test-passed result is generated, otherwise a test-failure result is generated. The reason, unless REASON-ARGS is provided, is generated based on the form of TEST: (predicate expected actual) - Means that we want to check whether, according to PREDICATE, the ACTUAL value is in fact what we EXPECTED. (predicate value) - Means that we want to ensure that VALUE satisfies PREDICATE. Wrapping the TEST form in a NOT simply produces a negated reason string.
  • Macro SKIP (&rest reason)
    Generates a TEST-SKIPPED result.
  • Macro IS-EVERY (predicate &body clauses)
    The input is either a list of lists, or a list of pairs. Generates (is (,predicate ,expr ,value)) for each pair of elements or (is (,predicate ,expr ,value) ,@reason) for each list.
  • Macro IS-TRUE (condition &rest reason-args)
    Like IS this check generates a pass if CONDITION returns true and a failure if CONDITION returns false. Unlike IS this check does not inspect CONDITION to determine how to report the failure.
  • Macro IS-FALSE (condition &rest reason-args)
    Generates a pass if CONDITION returns false, generates a failure otherwise. Like IS-TRUE, and unlike IS, IS-FALSE does not inspect CONDITION to determine what reason to give it case of test failure
  • Macro SIGNALS (condition-spec &body body)
    Generates a pass if BODY signals a condition of type CONDITION. BODY is evaluated in a block named NIL, CONDITION is not evaluated.
  • Macro FINISHES (&body body)
    Generates a pass if BODY executes to normal completion. In other words if body does signal, return-from or throw this test fails.
  • Macro PASS (&rest message-args)
    Simply generate a PASS.
  • Macro FAIL (&rest message-args)
    Simply generate a FAIL.
  • Function GET-FIXTURE (key &optional default)
  • Function (setf GET-FIXTURE) (value key)
  • Function REM-FIXTURE (key)
  • Macro DEF-FIXTURE (name (&rest args) &body body)
    Defines a fixture named NAME. A fixture is very much like a macro but is used only for simple templating. A fixture created with DEF-FIXTURE is a macro which can use the special macrolet &BODY to specify where the body should go. See Also: WITH-FIXTURE
  • Macro WITH-FIXTURE (fixture-name (&rest args) &body body)
    Insert BODY into the fixture named FIXTURE-NAME. See Also: DEF-FIXTURE
  • Macro FOR-ALL (bindings &body body)
    Bind BINDINGS to random variables and test BODY *num-trials* times. BINDINGS is a list of binding forms, each element is a list of (BINDING VALUE &optional GUARD). Value, which is evaluated once when the for-all is evaluated, must return a generator which be called each time BODY is evaluated. BINDING is either a symbol or a list which will be passed to destructuring-bind. GUARD is a form which, if present, stops BODY from executing when IT returns NIL. The GUARDS are evaluated after all the random data has been generated and they can refer to the current value of any binding. NB: Generator forms, unlike guard forms, can not contain references to the boud variables. Examples: (for-all ((a (gen-integer))) (is (integerp a))) (for-all ((a (gen-integer) (plusp a))) (is (integerp a)) (is (plusp a))) (for-all ((less (gen-integer)) (more (gen-integer) (< less more))) (is (<= less more))) (for-all (((a b) (gen-two-integers))) (is (integerp a)) (is (integerp b)))
  • Function GEN-INTEGER (&key (max (1+ most-positive-fixnum)) (min (1- most-negative-fixnum)))
    Returns a generator which produces random integers greater than or equal to MIN and less than or equal to MIN.
  • Function GEN-FLOAT (&key bound (type 'short-float))
    Returns a generator which producs floats of type TYPE. BOUND, if specified, constrains the ruselts to be in the range (-BOUND, BOUND).
  • Function GEN-CHARACTER (&key (code-limit char-code-limit) (code (gen-integer :min 0 :max (1- code-limit))) (alphanumericp nil))
    Returns a generator of characters. CODE must be a generator of random integers. ALPHANUMERICP, if non-NIL, limits the returned chars to those which pass alphanumericp.
  • Function GEN-STRING (&key (length (gen-integer :min 0 :max 80)) (elements (gen-character)) (element-type 'character))
    Returns a generator which producs random strings. LENGTH must be a generator which producs integers, ELEMENTS must be a generator which produces characters of type ELEMENT-TYPE.
  • Function GEN-LIST (&key (length (gen-integer :min 0 :max 10)) (elements (gen-integer :min -10 :max 10)))
    Returns a generator which producs random lists. LENGTH must be an integer generator and ELEMENTS must be a generator which producs objects.
  • Function GEN-TREE (&key (size 20) (elements (gen-integer :min -10 :max 10)))
    Returns a generator which producs random trees. SIZE control the approximate size of the tree, but don't try anything above 30, you have been warned. ELEMENTS must be a generator which will produce the elements.
  • Function GEN-BUFFER (&key (length (gen-integer :min 0 :max 50)) (element-type '(unsigned-byte 8)) (elements (gen-integer :min 0 :max (1- (expt 2 8)))))
  • Function GEN-ONE-ELEMENT (&rest elements)
  • Function GET-TEST (key &optional default)
  • Function (setf GET-TEST) (value key)
  • Function REM-TEST (key)
  • Function TEST-NAMES
  • Macro TEST (name &body body)
    Create a test named NAME. If NAME is a list it must be of the form: (name &key depends-on suite fixture compile-at profile) NAME is the symbol which names the test. DEPENDS-ON is a list of the form: (AND . test-names) - This test is run only if all of the tests in TEST-NAMES have passed, otherwise a single test-skipped result is generated. (OR . test-names) - If any of TEST-NAMES has passed this test is run, otherwise a test-skipped result is generated. (NOT test-name) - This is test is run only if TEST-NAME failed. AND, OR and NOT can be combined to produce complex dependencies. If DEPENDS-ON is a symbol it is interpreted as `(AND ,depends-on), this is accomadate the common case of one test depending on another. FIXTURE specifies a fixture to wrap the body in. If PROFILE is T profiling information will be collected as well.
  • Variable *DEFAULT-TEST-COMPILATION-TIME*
    :definition-time
  • Macro DEF-TEST (name (&key depends-on (suite '*suite* suite-p) fixture (compile-at *default-test-compilation-time*) profile) &body body)
    Create a test named NAME. NAME is the symbol which names the test. DEPENDS-ON is a list of the form: (AND . test-names) - This test is run only if all of the tests in TEST-NAMES have passed, otherwise a single test-skipped result is generated. (OR . test-names) - If any of TEST-NAMES has passed this test is run, otherwise a test-skipped result is generated. (NOT test-name) - This is test is run only if TEST-NAME failed. AND, OR and NOT can be combined to produce complex dependencies. If DEPENDS-ON is a symbol it is interpreted as `(AND ,depends-on), this is accomadate the common case of one test depending on another. FIXTURE specifies a fixture to wrap the body in. If PROFILE is T profiling information will be collected as well.
  • Variable *RUN-TEST-WHEN-DEFINED*
    nil
    When non-NIL tests are run as soon as they are defined.
  • Variable *VERBOSE-FAILURES*
    nil
    T if we should print the expression failing, NIL otherwise.
  • Generic-Function EXPLAIN (explainer results &optional stream recursive-depth)
    Given a list of test results report write to stream detailed human readable statistics regarding the results.
  • Method EXPLAIN ((exp detailed-text-explainer) results &optional (stream *test-dribble*) (recursive-depth 0))
  • Method EXPLAIN ((exp simple-text-explainer) results &optional (stream *test-dribble*) (recursive-depth 0))
  • Macro DEF-SUITE (name &key description in)
    Define a new test-suite named NAME. IN (a symbol), if provided, causes this suite te be nested in the suite named by IN. NB: This macro is built on top of make-suite, as such it, like make-suite, will overrwrite any existing suite named NAME.
  • Function MAKE-SUITE (name &key description ((in parent-suite)))
    Create a new test suite object. Overrides any existing suite named NAME.
  • Macro IN-SUITE (suite-name)
    Set the *suite* special variable so that all tests defined after the execution of this form are, unless specified otherwise, in the test-suite named SUITE-NAME. See also: DEF-SUITE *SUITE*
  • Macro IN-SUITE* (suite-name &key in)
    Just like in-suite, but silently creates missing suites.
  • Variable *ON-ERROR*
    nil
    The action to perform on error: - :DEBUG if we should drop into the debugger - :BACKTRACE to print a backtrace - NIL to simply continue
  • Variable *ON-FAILURE*
    nil
    The action to perform on check failure: - :DEBUG if we should drop into the debugger - :BACKTRACE to print a backtrace - NIL to simply continue
  • Variable *DEBUG-ON-ERROR*
    nil
    T if we should drop into the debugger on error, NIL otherwise. OBSOLETE: superseded by *ON-ERROR*
  • Variable *DEBUG-ON-FAILURE*
    nil
    T if we should drop into the debugger on a failing check, NIL otherwise. OBSOLETE: superseded by *ON-FAILURE*
  • Variable *PRINT-NAMES*
    t
    T if we should print test running progress, NIL otherwise.
  • Function RESULTS-STATUS (result-list)
    Given a list of test results (generated while running a test) return true if all of the results are of type TEST-PASSED, fail otherwise. Returns a second value, which is the set of failed tests.
  • Function RUN! (&optional (test-spec *suite*) &key ((print-names *print-names*) *print-names*))
    Equivalent to (explain! (run TEST-SPEC)).
  • Function EXPLAIN! (result-list)
    Explain the results of RESULT-LIST using a detailed-text-explainer with output going to *test-dribble*. Return a boolean indicating whether no tests failed.
  • Function DEBUG! (&optional (test-spec *suite*))
    Calls (run! test-spec) but enters the debugger if any kind of error happens.
  • Function RUN (test-spec &key ((print-names *print-names*) *print-names*))
    Run the test specified by TEST-SPEC. TEST-SPEC can be either a symbol naming a test or test suite, or a testable-object object. This function changes the operations performed by the !, !! and !!! functions.
  • Function !
    Rerun the most recently run test and explain the results.
  • Function !!
    Rerun the second most recently run test and explain the results.
  • Function !!!
    Rerun the third most recently run test and explain the results.
  • Function RUN-ALL-TESTS (&key (summary :end))
    Runs all defined test suites, T if all tests passed and NIL otherwise. SUMMARY can be :END to print a summary at the end, :SUITE to print it after each suite or NIL to skip explanations.