nst

API Reference

asdf-nst

NST integration with ASDF

SIFT.ASDF-NST

Unit and regression testing for Common Lisp
  • Function REPORT-SYSTEM (system &optional (stream nil stream-supp-t))

comp-set

No packages.

mnst-relay

Does not define NST tests, but runs the ones in a different system

No packages.

nst

The NST unit/regression testing system

NST

Unit and regression testing for Common Lisp
  • Variable *DEBUG-ON-ERROR*
    nil
    User variable =*debug-on-error*=: if non-nil, will break into the Lisp REPL debugger upon encountering an unexpected error. If nil, will record the error and continue with other tests.
  • Variable *DEBUG-ON-FAIL*
    nil
    User variable =*debug-on-fail*=: if non-nil, will break into the Lisp REPL debugger upon encountering a test which fails. If nil, will record the failure and continue with other tests. This variable is useful inspecting the dynamic environment under which a test was evaluated.
  • Variable *DEFAULT-REPORT-VERBOSITY*
    2
    User variable =*default-report-verbosity*= determines the default value for =*nst-verbosity*= when printing reports (2 by default).
  • Variable *NST-OUTPUT-STREAM*
    t
    Determines the output stream to which NST should print its output (=*standard-output*= by default).
  • Macro DEF-TEST-GROUP (group-name given-fixtures &body forms)
    The =def-test-group= form defines a group of the given name, providing one instantiation of the bindings of the given fixtures to each test. Groups can be associated with fixture sets, stateful initialization, and stateful cleanup. #+begin_example (def-test-group NAME (FIXTURE FIXTURE ...) (:aspirational FLAG) (:setup FORM FORM ... FORM) (:cleanup FORM FORM ... FORM) (:startup FORM FORM ... FORM) (:finish FORM FORM ... FORM) (:each-setup FORM FORM ... FORM) (:each-cleanup FORM FORM ... FORM) (:include-groups GROUP GROUP ... GROUP) (:documentation STRING) TEST ... TEST) #+end_example Arguments: - group-name :: Name of the test group being defined - given-fixtures :: List of the names of fixtures and anonymous fixtures to be used with the tests in this group. - aspirational :: An aspirational test is one which verifies some part of an API or code contract which may not yet be implemented. When a group is marked aspirational, all tests within the group are taken to be aspirational as well. At this point, there is no particular processing for aspirational tests and groups, but we hope to implement it at some point in the future. - forms :: Zero or more test forms, given by =def-check=. - setup :: These forms are run once, before any of the individual tests, but after the fixture names are bound. - cleanup :: These forms are run once, after all of the individual tests, but while the fixture names are still bound. - startup :: These forms are run once, before any of the individual tests and before the fixture names are bound. - finish :: These forms are run once, after all of the individual tests, and after the scope of the bindings to fixture names. - each-setup :: These forms are run before each individual test. - each-cleanup :: These forms are run after each individual test. - include-group :: The test groups named in this form will be run (respectively reported) anytime this group is run (reported). - documentation :: Docstring for the class.
  • Function CHECK-CRITERION-ON-VALUE (criterion val)
    The =check-criterion-on-value= function can be called from within a criterion body to verify that a value adheres to a criterion. #+begin_example (check-criterion-on-value CRITERION VALUE) #+end_example
  • Function CHECK-CRITERION-ON-FORM (criterion form)
    This function verifies that the values return by evaluating the form adheres to the criterion. #+begin_example (check-criterion-on-form CRITERION FORM) #+end_example
  • Macro DEF-CRITERION ((name args-formals values-formals) &body forms)
    The =def-criterion= macro defines a new criterion for use in NST tests. These criteria definitions are like generic function method definitions with two sets of formal parameters: the forms provided as the actual parameters of the criterion itself, and the values arising from the evaluation of the forms under test. #+begin_example (def-criterion (name criterion-lambda-list values-lambda-list) [ doc-string ) form form ... form) #+end_example - name :: Name of the criterion. - criterion-lambda-list :: Lambda list for the arguments to the criterion. Optionally, the first element of the list is a symbol specifying the parameter-passing semantics for the criterion arguments: =:values= for call-by-value, or =:forms for call-by-name (the default). The list may include the keywords =&key=, =&optional=, =&body= and =&rest= but may not use =&whole= or =&environment=. Apart from this restriction, in the former case the list may be any ordinary lambda list as for =defun=, and in the latter case the list may be any macro lambda list as for =defmacro=. - values-lambda-list :: Lambda list for the forms under test. Optionally, the first element of the list is a symbol specifying the parameter-passing semantics for the criterion arguments: :values for call-by-value (the default), or :form for call-by-name. In the former case, the list may include the keywords =&key=, =&optional=, =&body= and =&rest=, but not =&whole= or =&environment=; apart from that restriction, list may be any ordinary lambda list as for =defun=. In the latter case, the remainder of the list must contain exactly one symbol, to which a form which would evaluate to the values under test will be bound. If the criterion ignores the values, then instead of a lambda list, this argument may be the symbol =:ignore=. On many platforms, listing a dummy parameter which is then =declare=d =ignore= or =ignorable= will produce a style warning: the body of a =def-criterion= should not be assumed to correspond directly to the body of a =defmethod=; in general there will be surrounding =destructuring-bind=s. - documentation :: An optional documentation string for the criterion. - form :: The body of the criterion definition should return a test result report contructed with the =make-success-report=, etc. functions. Examples: #+begin_example (def-criterion (:true () (bool)) (if bool (make-success-report) (make-failure-report :format "Expected non-null, got: ~s" :args (list bool)))) (def-criterion (:eql (target) (actual)) (if (eql (eval target) actual) (make-success-report) (make-failure-report :format "Not eql to value of ~s" :args (list target)))) #+end_example
  • Macro DEF-CRITERION-UNEVALUATED ((name args-formals forms-formal &key (ignore-forms nil)) &body forms)
    The =def-criterion-unevaluated= macro is deprecated as of NST 2.1.2. It was consolidated into the =def-criterion= macro. Replace: #+begin_example (def-criterion-unevaluated name (pattern ... pattern) name BODY) #+end_example with: #+begin_example (def-criterion name (:forms pattern ... pattern) (:form name) BODY) #+end_example
  • Macro DEF-VALUES-CRITERION ((name args-formals forms-formals &key (declare nil decl-supp-p)) &body forms)
    The =def-values-criterion= macro was deprecated as of NST 1.3.0. For new criteria, use =def-criterion= instead. In the short term, code using =def-values-criterion= should continue to work as before.
  • Macro DEF-FORM-CRITERION ((name args-formals form-formal) &rest forms)
    The =def-form-criterion= macro was deprecated as of NST 1.3.0. /Code using =def-form-criterion= in any but the simplest ways is very likely to fail./ Use =def-criterion= instead.
  • Macro DEF-CRITERION-ALIAS ((name . args-formals) docstring-or-form &optional (form nil form-supp-p))
    The simplest mechanism for defining a new criterion involves simply defining one criterion to rewrite as another using =def-criterion-alias=. #+begin_example (def-criterion-alias (name (:seq arg)) [ doc-string ] expansion) #+end_example The body of the expansion should be a Lisp form which, when evaluated, returns an S-expression quoting the new criterion which the rewrite should produce. The =arg= are passed as for Lisp macros: they are not evaluated and are most typically comma-inserted into a backquoted result. For example: #+begin_example (def-criterion-alias (:forms-eq) `(:predicate eq)) (def-criterion-alias (:symbol name) `(:eq ',name)) #+end_example
  • Macro DEF-FIXTURES (name (&key (uses nil uses-supp-p) (assumes nil assumes-supp-p) special outer inner documentation cache setup cleanup startup finish export-names (export-bound-names nil export-bound-names-supp-p) (export-fixture-name nil export-fixture-name-supp-p)) &body bindings)
    Fixtures are data structures and values which may be referred to by name during testing. NST provides the ability to use fixtures across multiple tests and test groups, and to inject fixtures into the runtime namespace for debugging. A set of fixtures is defined using the =def-fixtures= macro: #+begin_example (def-fixtures fixture-name ( [ :outer FORM ] [ :inner FORM ] [ :setup FORM ] [ :cleanup FORM ] [ :startup FORM ] [ :finish FORM ] [ :documentation STRING ] [ :cache FLAG ] [ :export-names FLAG ] [ :export-fixture-name FLAG ] [ :export-bound-names FLAG ] ) ( [ ( [ :cache FLAG ] ) ] NAME [ FORM ] ) ... ( [ ( [ :cache FLAG ] ) ] NAME [ FORM ] ) ) #+end_example - fixture-name :: The name to be associated with this set of fixtures. - inner :: List of declarations to be made inside the let-binding of names of any use of this fixture. Do not include the "declare" keyword here; NST adds these declarations to others, including a special declaration of all bound names. - outer :: List of declarations to be made outside the let-binding of names of any use of this fixture. - documentation :: A documentation string for the fixture set. - special :: Specifies a list of names which should be declared =special= in the scope within which this set's fixtures are evaluated. The individual names are taken to be single variable names. Each =(:fixture NAME)= specifies all of the names of the given fixture set. This declaration is generally optional under most platforms, but can help supress spurious warnings. Note that multiple =NAMEs= may be listed, and these lists and the bare names may be intermixed. If only one name or fixture is specified, it need not be placed in a list - export-fixture-name :: When non-nil, the fixture name will be added to the list of symbols exported by the current package. - export-bound-names :: When non-nil, the names bound by this fixture will be added to the list of symbols exported by the current package. - export-names :: When non-nil, sets the default value to t for the two options above. - cache :: If specified with the group options, when non-nil, the fixture values are cached at their first use, and re-applied at subsequent fixture application rather than being recalculated. When a fixture is attached to a test or test group, each =NAME= defined in that fixture becomes available in the body of that test or group as if =let*= bound to the corresponding =FORM=. A fixture in one set may refer back to other fixtures in the same set (again as =let*=) but forward references are not allowed. The four arguments =startup=, =finish=, =setup= and =cleanup= specify forms which are run everytime the fixture is applied to a group or test. The =startup= (respectively =finish=) form is run before fixtures are bound (after their bindings are released). These forms are useful, for example, to initialize a database connection from which the fixture values are drawn. The =setup= form is run after inclusion of names from fixture sets, but before any tests from the group. The =cleanup= form is normally run after the test completes, but while the fixtures are still in scope. Normally, the =cleanup= form will not be run if the =setup= form raises an error, and the =finish= form will not be run if the =startup= form raises an error; although the user is able to select (perhaps unwisely) a restart which disregards the error. The names of a fixture and the names it binds can be exported from the package where the fixture is defined using the =export-bound-names= and =export-fixture-name= arguments. The default value of both is the value of =export-names=, whose default value is =nil=. The =cache= option, if non-nil, directs NST to evaluate a fixture's form one single time, and re-use the resulting value on subsequent applications of the fixture. Note that if this value is mutated by the test cases, test behavior may become unpredictable! However this option can considerably improve performance when constant-valued fixtures are applied repeatedly. Caching may be set on or off (the default is off) for the entire fixture set, and the setting may vary for individual fixtures. Examples of fixture definitions: #+begin_example (def-fixtures f1 () (c 3) (d 'asdfg)) (def-fixtures f2 (:special ((:fixture f1))) (d 4) (e 'asdfg) (f c)) (def-fixtures f3 () ((:cache t) g (ackermann 1 2)) ((:cache nil) h (factorial 5))) #+end_example To cause a side-effect among the evaluation of a fixture's name definitions, =nil= can be provided as a fixture name. In uses of the fixture, NST will replace =nil= with a non-interned symbol; in documentation such as form =:whatis=, any =nil=s are omitted.
  • Macro WITH-FIXTURES ((&rest fixtures) &body forms)
    The =with-fixtures= macro faciliates debugging and other non-NST uses of fixtures sets: #+begin_example (with-fixtures (FIXTURE FIXTURE ... FIXTURE) FORM FORM ... FORM) #+end_example This macro evaluates the forms in a namespace expanded with the bindings provided by the fixtures.
  • Macro DEF-TEST (name-or-name-and-args criterion &rest forms)
    Individual unit tests are encoded with the =def-test= form: #+begin_example (def-test NAME ( [ :group GROUP-NAME ] [ :setup FORM ] [ :cleanup FORM ] [ :startup FORM ] [ :finish FORM ] [ :fixtures (FIXTURE FIXTURE ... FIXTURE) ] [ :aspirational FLAG ] [ :documentation STRING ] ) criterion FORM ... FORM) (def-test NAME criterion &body (:seq FORM)) #+end_example The =SETUP=, =CLEANUP=, =STARTUP=, =FINISH= and =FIXTURES= are just as for fixtures and test groups, but apply only to the one test. The =CRITERION= is a list or symbol specifying the properties which should hold for the =FORM=s. When a test is not enclosed within a group body, a group name must be provided by the =GROUP= option. When a test is enclosed within a group body, the =GROUP= option is not required, but if provided it must agree with the group name. When there are no =SETUP=, =CLEANUP=, =STARTUP=, =FINISH= or =FIXTURES= arguments, the =NAME= may be given without parentheses. Likewise, any criterion consisting of a single symbol, e.g. =(:pass)=, may be abbreviated as just the symbol without the parentheses, e.g. =:pass=. The =:documentation= form provides a documentation string in the standard Lisp sense. Since documentation strings are stored against names, and since the same name can be used for several tests (so long as they are all in different packages), documentation strings on tests may not be particularly useful. An =aspirational= test is one which verifies some part of an API or code contract which may not yet be implemented. When a group is marked aspirational, all tests within the group are taken to be aspirational as well. At this point, there is no particular processing for aspirational tests and groups, but we hope to implement it at some point in the future. The =def-check= form is a deprecated synonym for =def-test=.
  • Function MAKE-ERROR-REPORT (e &rest format-args)
    Function =make-error-report= produces a report of an error during test execution. #+begin_example (make-error-report [ :format FORMAT-STRING ] [ :args ARG-FORM-LIST ] ) #+end_example
  • Function WRAP-THROWN-LISP-WARNING (w)
    The helper function =wrap-thrown-lisp-warning= creates an NST =check-note= object from a standard Lisp =warning=.
  • Generic-Function ADD-WARNING (result w &rest args)
    For use within user-defined NST criteria: add a warning to a result. The =add-warning= function adds an warning to a result record. The item can be any of a Lisp warning, an NST check-note or a format string; in the first two cases, no additional arguments should be provided. #+begin_example (add-warning RESULT-REPORT [ :format ITEM ] [ :args ARGUMENT-LIST ] ) #+end_example
  • Method ADD-WARNING (result (w warning) &rest args)
  • Method ADD-WARNING (result (w warning) &rest args)
  • Method ADD-WARNING (result (w warning) &rest args)
  • Method ADD-WARNING (result (w check-note) &rest args)
  • Method ADD-WARNING (result (format-string string) &rest args)
  • Function MAKE-WARNING-REPORT (&key format args)
    Function =make-warning-report= is like =make-failure-report=, but provides supplimentary information as a warning. #+begin_example (make-warning-report [ :format FORMAT-STRING ] [ :args ARG-FORM-LIST ]) #+end_example The =emit-warning= function is an older, deprecated version of this function.
  • Macro EMIT-WARNING (&rest args)
    The =emit-warning= function is deprecated; use =make-warning-report= instead.
  • Function MAKE-FAILURE-REPORT (&key format args info)
    The =make-failure-report= function returns a report of test failure. #+begin_example (make-failure-report FORMAT ARGS) #+end_example The =format-string= and =args= are as to the Common Lisp function =format=. The =emit-failure= function is an older, deprecated version of this function.
  • Macro EMIT-FAILURE (&rest args)
    The =emit-failure= function is deprecated; use =make-failure-report= instead.
  • Function MAKE-SUCCESS-REPORT (&rest args &key warnings info)
    The =make-success-report= function indicates a successful test result. #+begin_example (make-success-report) #+end_example Note that some older examples show =(make-check-result)=, =(emit-success)= or =(check-result)=. The former is an internal function and should not be used from outside the core NST files. The latter two are deprecated.
  • Macro EMIT-SUCCESS (&rest args)
    The =emit-success= function is deprecated; use =make-success-report= instead.
  • Function ADD-FAILURE (result &key format args)
    For use within user-defined NST criteria: add a failure to a result. #+begin_example (add-failure RESULT-REPORT [ :format FORMAT-STRING ] [ :args ARGUMENT-LIST ]) #+end_example
  • Function ADD-ERROR (result &key format args)
    For use within user-defined NST criteria: add an error to a result. #+begin_example (add-error RESULT-REPORT [ :format FORMAT-STRING ] [ :args ARGUMENT-LIST ] ) #+end_example
  • Function ADD-INFO (result item)
    For use within user-defined NST criteria: add an info note to a result. #+begin_example (add-info RESULT-REPORT INFO-ITEM) #+end_example
  • Macro ASSERT-CRITERION (key-args criterion-expr &rest value-exprs)
    The =assert-criterion= macro asserts that an NST criterion should pass. #+begin_example (assert-criterion ( [ :msg-format FORMAT-STRING ] [ :msg-args FORMAT-ARGUMENTS ] [ :fatal FLAG [ [ :fail-on-warning FLAG ] ) FORM ... FORM) #+end_example - msg-format :: Format string used to build the label of the restart point. - msg-args :: Format arguments used to build the label of the restart point. - fatal :: If non-nil, a failure of this assertion indicates that execution of the test forms should be aborted. - fail-on-warning :: If non-nil, then an NST result which includes a warning indicates failure of this assertion.
  • Macro DEF-UNARY-PREDICATE-ASSERT (assert-fn predicate default-message &key (message-defvar nil defvar-supp-p) (doc-state-flag t) (pred-name predicate) &allow-other-keys)
    Macro =macro def-unary-predicate-assert= creates an assertion function using the result of a call to a unary predicate. A non-nil result from the predicate corresponds to a successful assertion. #+begin_example (def-unary-predicate-assert ASSERT-FN PREDICATE DEFAULT-MESSAGE [ :message-defvar NAME ] [ :pred-name NAME ] [ :doc-state-flag BOOL ] ) #+end_example - assert-fn :: The name of the assertion function being defined. - predicate :: The predicate used to define the assertion function. It should take a single argument. - default-message :: Format string used by default for reporting failures of this assertion. It should expect to be used in a call to =format= with one additional argument, the value being tested. - message-defvar :: The name of a global variable into which the default message will be stored. If this argument is omitted, the result of a call to =gensym= is used. - pred-name :: This argument is used only for documenting the underlying predicate in the assertion function's docstring. By default, it is the same as the predicate.
  • Macro DEF-UNARY-NEGATED-PREDICATE-ASSERT (assert-fn predicate default-message &rest keyargs &key message-defvar)
    Macro =def-unary-negated-predicate-assert= uses the negated result of a unary predicate as the basis of an assertion function. This macro's arguments are just as for =def-unary-predicate-assert=.
  • Macro DEF-BINARY-PREDICATE-ASSERT (assert-fn predicate default-message &key (message-defvar nil defvar-supp-p) (doc-state-flag t) (pred-name predicate))
    Macro =def-binary-predicate-assert= uses a binary predicate as the basis for an assertion function just as =def-unary-predicate-assert= uses a unary predicate. This macro's arguments are just as for =def-unary-predicate-assert=.
  • Macro DEF-BINARY-NEGATED-PREDICATE-ASSERT (assert-fn predicate default-message &rest keyargs &key message-defvar)
    Macro =def-binary-negated-predicate-assert= uses the negated result of a binary predicate as the basis for an assertion function just as =def-unary-negated-predicate-assert= uses the negated result of a unary predicate. This macro's arguments are just as for =def-unary-predicate-assert=.
  • Variable *ASSERT-NULL-FORMAT-STRING*
    "~@<expected null, ~_got ~s~:>"
  • Function ASSERT-NULL (g314 &rest g315 &key (format nil g316) (format-args nil g317) (fatal nil) &allow-other-keys)
    The =ASSERT-NULL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It succeeds whenever the =NULL= function returns non-nil.
  • Variable *ASSERT-ZERO-FORMAT-STRING*
    "~@<expected zero, ~_got ~s~:>"
  • Function ASSERT-ZERO (g360 &rest g361 &key (format nil g362) (format-args nil g363) (fatal nil) &allow-other-keys)
    The =ASSERT-ZERO= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It succeeds whenever the =ZEROP= function returns non-nil.
  • Variable *ASSERT-NONNIL-FORMAT-STRING*
    "~@<expected non-nil, ~_got ~s~:>"
  • Function ASSERT-NON-NIL (g406 &rest g407 &key (format nil g408) (format-args nil g409) (fatal nil) &allow-other-keys)
    The =ASSERT-NON-NIL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It succeeds whenever the =NULL= function returns =nil=.
  • Variable *ASSERT-EQ-FORMAT-STRING*
    "~@<expected eq to ~s, ~_got ~s~:>"
  • Function ASSERT-EQ (g452 g453 &rest g454 &key (format nil g455) (format-args nil g456) (fatal nil) &allow-other-keys)
    The =ASSERT-EQ= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQ=, succeeds whenever that call returns non-nil.
  • Variable *ASSERT-EQL-FORMAT-STRING*
    "~@<expected eql to ~s, ~_got ~s~:>"
  • Function ASSERT-EQL (g500 g501 &rest g502 &key (format nil g503) (format-args nil g504) (fatal nil) &allow-other-keys)
    The =ASSERT-EQL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQL=, succeeds whenever that call returns non-nil.
  • Variable *ASSERT-EQUAL-FORMAT-STRING*
    "~@<expected equal to ~s, ~_got ~s~:>"
  • Function ASSERT-EQUAL (g548 g549 &rest g550 &key (format nil g551) (format-args nil g552) (fatal nil) &allow-other-keys)
    The =ASSERT-EQUAL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQUAL=, succeeds whenever that call returns non-nil.
  • Variable *ASSERT-EQUALP-FORMAT-STRING*
    "~@<expected equalp to ~s, ~_got ~s~:>"
  • Function ASSERT-EQUALP (g596 g597 &rest g598 &key (format nil g599) (format-args nil g600) (fatal nil) &allow-other-keys)
    The =ASSERT-EQUALP= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQUALP=, succeeds whenever that call returns non-nil.
  • Variable *ASSERT-NOT-EQ-FORMAT-STRING*
    "~@<expected non-eq to ~s, ~_got ~s~:>"
  • Function ASSERT-NOT-EQ (g644 g645 &rest g646 &key (format nil g647) (format-args nil g648) (fatal nil) &allow-other-keys)
    The =ASSERT-NOT-EQ= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQ=, succeeds whenever that call returns =nil=.
  • Variable *ASSERT-NOT-EQL-FORMAT-STRING*
    "~@<expected non-eql to ~s, ~_got ~s~:>"
  • Function ASSERT-NOT-EQL (g692 g693 &rest g694 &key (format nil g695) (format-args nil g696) (fatal nil) &allow-other-keys)
    The =ASSERT-NOT-EQL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQL=, succeeds whenever that call returns =nil=.
  • Variable *ASSERT-NOT-EQUAL-FORMAT-STRING*
    "~@<expected non-equal to ~s, ~_got ~s~:>"
  • Function ASSERT-NOT-EQUAL (g740 g741 &rest g742 &key (format nil g743) (format-args nil g744) (fatal nil) &allow-other-keys)
    The =ASSERT-NOT-EQUAL= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQUAL=, succeeds whenever that call returns =nil=.
  • Variable *ASSERT-NOT-EQUALP-FORMAT-STRING*
    "~@<expected non-equalp to ~s, ~_got ~s~:>"
  • Function ASSERT-NOT-EQUALP (g788 g789 &rest g790 &key (format nil g791) (format-args nil g792) (fatal nil) &allow-other-keys)
    The =ASSERT-NOT-EQUALP= function is a unary predicate for use within the forms evaluated for an =:eval= criterion. It compares the expected and tested values using =EQUALP=, succeeds whenever that call returns =nil=.
  • Macro DEF-EVAL-TEST (name-or-name-and-args &rest forms)
    The =def-eval-test= macro abbreviates a call to =def-test= with a single =:eval= criterion. Its arguments are just as for =def-test= and =:eval=. #+begin_example (def-eval-test (NAME [ :group GROUP-NAME ] [ :setup FORM ] [ :cleanup FORM ] [ :startup FORM ] [ :finish FORM ] [ :fixtures (FIXTURE ... FIXTURE) ] [ :documentation STRING ] [ :check-warnings FLAG ] [ :muffle-warnings FLAG ] [ :attempt-continue FLAG ] [ :force-continue FLAG ] ) FORM ... FORM) (def-eval-test NAME FORM ... FORM) #+end_example
  • Macro NST-CMD (&rest args)
    User-level NST operations are accessible from the REPL via the =nst-cmd= macro. #+begin_example (nst-cmd NST-COMMAND ARG ... ARG) #+end_example Where a particular system supports the facility (Currently Allegro, and SBCL under ACL-REPL) the top-level alias =:nst= provides a shorthand to this function.
  • Generic-Function NST-JUNIT-DUMP (stream)
    Function =nst-junit-dump= pushes the entire NST state to a JUnit XML file whose stream is specified by its argument.
  • Method NST-JUNIT-DUMP ((stream stream))
  • Method NST-JUNIT-DUMP ((filename string))
    Write the results of this session's NST testing in JUnit XML format.
  • Function JUNIT-RESULTS-BY-GROUP (&rest args &key verbose &allow-other-keys)
    The \texttt{junit-results-by-group} function writes the NST test results in JUnit XML format, organized by group, aligning test groups with Java classes, and individual tests with \texttt{@Test} methods. #+begin_example (junit-results-by-group [ :verbose FLAG ] [ :dir DIRECTORY ] [ :file FILESPEC ] [ :stream STREAM ] [ :if-dir-does-not-exist BOOL ] [ :if-file-exists BOOL ] ) #+end_example Either =:dir= and =:file= options, or the =:stream= option, but not both, should e used to specify the target for XML output; if none of the three options are given, the function will write to =*standard-output*=.
  • Variable *MAX-COMPOUND-STRUCTURE-DEPTH*
    4
    The =*max-compound-structure-depth*= variable sets the maximum nesting depth of compound data structures: beyond that depth, =scalar= rather than =t= is the default element generator. This restriction does not apply to explicitly specified element types, only to the use of defaults.
  • Macro COMPOUND-STRUCTURE (&body forms)
    The =compound-structure= macro wraps substructure which should be considered compound for the limits set by =*max-compound-structure-depth*=.
  • Generic-Function ARBITRARY (typ)
    This function takes a single argument, which determines the type of the value to be generated. For simple types, the name of the type (or the class object, such as returned by =find-class= by itself is a complete specification. For more complicated types, =arbitrary= can also take a list argument, where the first element gives the type and the remaining elements are keyword argument providing additional requirements for the generated value.
  • Method ARBITRARY (other)
  • Method ARBITRARY (other)
  • Method ARBITRARY (other)
  • Method ARBITRARY ((spec cons))
  • Method ARBITRARY ((spec cons))
  • Method ARBITRARY ((spec cons))
  • Macro DEF-ARBITRARY-INSTANCE-TYPE (type-expr &body technique)
    New type specifications for invariant-testing. are defined with the =def-arbitrary-instance-type= macro. #+begin_example (def-arbitrary-instance-type ( SPEC-NAME [ :params FORMALS ] [ :scalar BOOL ] [ :key KEY ] ) FORM FORM ... FORM) #+end_example - =formals= :: Formal parameter definition used to pass subcomponent types. - =scalar= :: When a non-null value is provided for the =:scalar= argument, the new specifier is taken to be generable by the =:scalar= specification. #+begin_example (def-arbitrary-instance-type (ratio :scalar t) (/ (arbitrary 'integer) (let ((raw (arbitrary (find-class 'integer)))) (cond ((< raw 0) raw) (t (+ 1 raw)))))) #+end_example - =key= :: The =:key= argument gives a list of keyword arguments which may accompany the new specification. For the =cons= type, keyword arguments allow specifications for the left and right components: #+begin_example (def-arbitrary-instance-type (cons :key ((car t car-supp-p) (cdr t cdr-supp-p))) (compound-structure (when (and (not car-supp-p) (>= *current-compound-structure-depth* *max-compound-structure-depth*)) (setf car 'scalar)) (when (and (not cdr-supp-p) (>= *current-compound-structure-depth* *max-compound-structure-depth*)) (setf cdr 'scalar)) (cons (arbitrary car) (arbitrary cdr)))) #+end_example - =form= :: Construct and return (as if through =progn= the arbtrary instance.
  • Macro DEF-TEST-GENERIC (function-name &body forms)
    The =def-test-generic= declares a generic test function. #+begin_example (def-test-generic FUNCTION-NAME) #+end_example For example, #+begin_example (nst:def-test-generic for-clses) #+end_example
  • Macro DEF-TEST-METHOD (function-name (arg class) &body body)
    The =def-test-method= defines a general method for a generic test function. #+begin_example (def-test-method FUNCTION-NAME (TEST-VALUE CLASS-NAME) FORM ... FORM) #+end_example - function-name :: The name of the test function for which we are defining a method. - test-value :: Formal parameter to which the value under test will be bound. - class-name :: The class for which we are defining a method. The method body should return a test result report, constructed with =make-success-result=, etc. For example: #+begin_example (nst:def-test-method for-clses (o mid-cls) (with-slots (mc1 mc2) o (cond ((< mc1 mc2) (make-success-report)) (t (make-failure-report :format "~d not < ~d" :args (list mc1 mc2)))))) (nst:def-test-method for-clses (o side-cls) (with-slots (sc1 sc2) o (cond ((eql sc1 sc2) (make-success-report)) (t (make-failure-report :format "~d not eql ~d" :args (list sc1 sc2)))))) #+end_example
  • Macro DEF-TEST-METHOD-CRITERION (function-name class documentation &optional (criterion nil crit-supp-p))
    The =def-test-method-criterion= macro provides a simple facility for defining a generic test function method in terms of an NST criterion. #+begin_example (def-test-method-criterion FUNCTION-NAME CLASS-NAME [ CRITERION ] ) #+end_example - function-name :: The name of the test function for which we are defining a method. - class-name :: The class for which we are defining a method. - criterion :: The criterion to be applied to members of the class. For example: #+begin_example (nst:def-test-method-criterion for-clses top-cls (:predicate (lambda (tc) (< (tc1 tc) (tc2 tc))))) #+end_example

NST-NAME-USE-IN-PACKAGES

Internal package for names' record-keeping.
No exported symbols.

NST-NAME-USAGE

No exported symbols.

NST-ARTIFACT-LOOKUP-PACKAGE

Auxiliary package for canonical NST command names.
No exported symbols.

NST-CONTROL-API

API-level control of test execution and reporting.
No exported symbols.

Also exports

  • NST:REPORT-TEST
  • NST:REPORT-MULTIPLE
  • NST:PROTECT-NST-CONFIG
  • NST:RUN-PACKAGE
  • NST:REPORT-PACKAGE
  • NST:REPORT-GROUP
  • NST:APPLY-DEBUG-OPTIONS
  • NST:RUN-GROUP
  • NST:RUN-TEST

nst-manual-tests

Utilities for hand-run NST tests.

NST-MANUAL

Manually-run tests and utilities for NST
No exported symbols.

nst-meta-tests

M as in meta: NST- (or otherwise) testing NST.

NST-META

Package for reflective NST test suites
No exported symbols.

NST-META-SOURCES

Package for the source test suites for reflective NST tests
No exported symbols.

NST-META-SOURCES-1

Additional package for the source test suites for reflective NST tests
No exported symbols.

NST-METHODS-META-SOURCES

Package for the source test suites for reflective NST tests
No exported symbols.

nst-mop-utils

Utilities for NST which require the MOP.

NST-MOP-UTILS

MOP-based utilities for NST
  • Macro DEF-DERIVED-CRITERION (criterion &key (class nil class-supp-p) (slot-keys nil) (ignore-slots nil))

nst-selftest-utils

Cmmon utilities for NST self-tests.

NST-TEST-UTILS

Package for common NST test utilities.
No exported symbols.

nst-test

M as in meta: NST- (or otherwise) testing NST.

No packages.

nst-test-jenkins

Test NST, and raise an error if tests do not pass.

No packages.