clod

API Reference

clod

Common Lisp Autodoc generator

CLOD

* Description

CLOD is a tool for creating documentation for Common Lisp programs.
CLOD examines a loaded package and writes information about all the
symbols defined within that package, to a file.

The output file is in /Org/ format. Org is a simple but powerful 'wiki-like'
markup language that is understood by *Org-mode*, a powerful outliner, personal
wiki and organiser that runs as a major mode within the /Emacs/ text
editor. Org-mode can export org-format files to numerous other formats,
including HTML, LaTeX, PDF, DocBook, and plain text.

More information:
- Emacs :: [[http://www.gnu.org/software/emacs]] (if you program CL, you won't
  need to be told what Emacs is)
- Org mode :: [[http://orgmode.org/]]

* Why use CLOD?

- You can use org markup within docstrings (easy for humans to read) to
  create subsections, bulleted lists, hyperlinks within the document or
  to external URLs, etc.
- Easy export to multiple file formats: text, HTML, DocBook, LaTeX -> PDF...
- You can edit the output within Emacs, which is already the IDE of most
  Commn Lisp programmers.
- If GraphViz is installed, automatically create a diagram illustrating the
  package's class hierarchy -- see http://www.graphviz.org/
- Org markup has many cool tricks. Some examples of useful things you can
  put in docstrings:
  - Include an entire file and format as plain text:
    : #+INCLUDE: "filename" quote
  - Include an entire file with common lisp syntax highlighting
    : #+INCLUDE: "filename.lisp" src lisp
  - Timestamps and datestamps:
    : {{{ modification-time(%r) }}}
    {{{modification-time(%r)}}}
    : {{{ date(%Y-%m-%d) }}}
    {{{date(%Y-%m-%d)}}}
  - Define text macros, use with ={{{macro(arg1,arg2)}}}=
    : #+MACRO: foo Replacement text $1 $2
  - Embed any LaTeX code directly (no special markup needed)
  - Embed any HTML code (special markup needed)
  - Automatic syntax highlighting of source code examples in exported
    documents, eg:
;;; (defun hello (a &key b)
;;;    (print "Hello, world!"))
  - Because CLOD works by introspection rather than parsing files,
    it copes effortlessly with unusual toplevel forms. Documentation
    generators that parse files usually won't be able to tell that your
    toplevel macro `=(defclass-easy ...)=' (or whatever) actually
    expands to a class definition.
  - For the same reason, CLOD avoids the problems with dependencies that
    can trouble other documentation generators (i.e. difficulties
    generating correct documentation unless you manually move things
    around in the source so that the doc generator finds things in
    the `right' order).
  - Easily change the global appearance of the document by specifying a
    cascading style sheet (/note: only affects HTML export from Org mode/)

* Dependencies

- ASDF: [[http://common-lisp.net/project/asdf/]]
- Closer-MOP: [[http://common-lisp.net/project/closer/]]
- Iterate: [[http://common-lisp.net/project/iterate/]]
- CL-PPCRE: [[http://weitz.de/cl-ppcre/]]

* How to use

1. Install CLOD somewhere ASDF can find it.
2. Load CLOD with =(asdf:oos 'asdf:load-op :clod)=
3. Load the package or packages for which you wish to produce documentation,
   eg: =(asdf:oos 'asdf:load-op :mypkg)=
4. Run =(clod:document-package :mypkg nil)= and you will see the documentation
   as a string, returned by [[document-package]].
5. Run =(clod:document-package :mypkg "filename.org")= and the same
   documentation will be written to 'filename.org'.
6. Load filename.org into Emacs. Export to HTML with =M-x org-export=, or press
   C-c C-e to be presented with a menu of export options.

* Writing the documentation

All documentation produced by CLOD is obtained by /introspection/, i.e. by the
running CL process examining itself. If a symbol has an associated docstring,
it will be used by CLOD to produce the documentation for that symbol.

Within documentation strings, you can use org mode markup. This is a simple,
human-readable markup language similar to the type of markup used for wiki
editing and forum posting. See the Org
[[http://orgmode.org/manual/Markup.html][manual]] for more information. Also see
the docstrings in the CLOD source code, which use org markup.

Some special points to note:
- Outline headings are used to structure org documents. These headings
  begin with one or more asterisks at the start of the line. Thus, if you
  want a large docstring to be divided into subsections, the heading for
  each subsection should be a line that starts with one or more asterisks (*),
  then a space, then the title of the heading.
- These headings will automatically be correctly 'indented' for their location
  in the structure of the final document. The whole document is one outline,
  and any given docstring will usually be appearing 2 or 3 levels deep within
  that outline. However, CLOD finds all heading lines within docstrings and
  increases the number of asterisks appropriately.
- An extra blank line is also automatically inserted after headings within
  docstrings, allowing you to save screen space in the docstring itself.
- By default, many docstrings are inserted within a subsection titled
  'Description'. However, if you don't want this to happen, but rather want
  the docstring to define its own heading names, make sure that the very first
  thing in the docstring is a heading (straight after the opening quote).
  (Note for mmm-mode users (see below): if the docstring starts with '###'
  to signal that it is in fact a docstring, CLOD will skip the hashes before
  looking to see if the string starts with a heading.)
  So ="###* Arguments ..."= will work in that case.
- Some symbol names used by common lisp can conflict with the markup used
  by org mode. For example, =*global-variable*=: asterisks are interpreted
  by org mode as signifying bold text. CLOD catches these in headings and
  auto-generated documentation, but not within doc strings, where you will
  need to surround the offending symbol with =equals signs=.
- *Hyperlinks* are created using
  : [[double square brackets]]
  Any text surrounded by these brackets will link to the same text (case
  insensitive) surrounded by =<<double angle brackets>>=. CLOD uses this to
  define hyperlinks for all symbols in the package. Every symbol MYSYMBOL has:
  1. A hyperlink =<<function MYSYMBOL>>= if MYSYMBOL is a function,
     =<<variable MYSYMBOL>>= if it is a global variable, etc.
  2. A hyperlink =<<MYSYMBOL>>= which will either link to MYSYMBOL's
     documentation, or to a 'disambiguation section' if the same symbol has
     multiple meanings (eg there is both a function and a variable called
     MYSYMBOL).
- Org mode has the ability to use Emacs' font-lock mode to produce source code
  snippets that are correctly syntax highlighted for any major mode.  To use
  this normally requires surrounding the code with =#+BEGIN_SRC ... #+END_SRC=.
  CLOD provides a shortcut: Any lines within docstrings that begin with three
  semicolons =;;;= are assumed to be example lisp source code. The first 3
  semicolons are removed and the rest of the line is syntax highlighted.

* Combining org mode and common lisp mode in a single Emacs buffer

You can use org mode markup within docstrings, but you can't see the effects of
the markup until you export the documentation to org using CLOD. You also don't
get access to org's support for automatic formatting of bulleted lists as you
write, or the fantastic support for writing tables, or hyperlinks that you can
click with the mouse, or ....

What if you could use all the goodness of Org, while editing docstrings in your
lisp source code? You can. This section explains how.

1. Download and install nXhtml, an emacs package that contains code allowing
   multiple major modes to be active in a single buffer.
   http://ourcomments.org/cgi-bin/emacsw32-dl-latest.pl
2. Add the code in `mmm-clod.el' to your .emacs file. Make sure you change
   the mmm-mode directory to the directory where you installed mmm-mode.
3. Restart emacs. Load a lisp source file. All documentation strings should
   appear with a coloured background, and when you move the cursor inside them,
   you will see 'Lisp[Org]' on the modeline.
4. If not everything is highlighting correctly, or if you write a new docstring
   and org does not activate within it, press control-` to 'refresh' mmm-mode.

Not everything works: expanding and collapsing headings fails, and
clicking the mouse elsewhere within the doc string often causes problems. But
overall the two modes work together surprisingly well.

MMM-mode recognises the following things as doc strings:
1. Any string that emacs fontifies using 'font-lock-doc-face'. (in other words,
   font-lock mode must be active.)
2. Any string inside the form '=(:documentation STRING)='.
3. Finally, any string whose first three characters are '###'. Since lines
   beginning with a hash are interpreted as comments by org mode, these
   characters will disappear when you export your document to HTML or other
   formats.

* Example docstring

Here is the docstring for [[document-package]]. It illustrates the use of
headings, bulleted lists, definition lists, =code=, *bold* and /italic/
markup, hyperlinks to other definitions, and syntax highlighting of lisp source
code examples.

: "* Arguments
: - PKG :: A package name or package object.
: - FILE/STREAM :: A string (filename), stream object, or =NIL=.
: - AUTO-LINKS :: Boolean.
: - LINES-BETWEEN-SECTIONS :: Boolean.
: - BRIEF-METHODS :: Boolean.
: - STYLE-SHEET :: A string.
: - TITLE :: A string.
: - AUTHOR :: A string.
: - EMAIL :: A string.
: * Returns
: A string, or nil.
: * Description
: Produce documentation for the package =PKG=.
:
: The documentation's destination depends on the value of =FILE/STREAM=:
: - =STRING=: documentation is written to the file named by the string.
: - =STREAM=: documentation is written to the already existing stream.
: - =NIL=: documentation is written to a string, which is then returned by
:   this function.
:
: Explanation of optional arguments:
: - =TITLE=, =AUTHOR= and =EMAIL= specify the document title, the name of
:   the author, and the email address of the author.
: - If =AUTO-LINKS= is non-nil, *all* occurrences of symbol names within the
:   text of docstrings will be interpreted as hyperlinks, regardless of
:   whether they are marked up as hyperlinks.
: - If LINES-BETWEEN-SECTIONS is nil, do not output a horizontal line before
:   each new section of documentation.
: - If BRIEF-METHODS is nil, document individual methods with their own
:   sections, just like functions and generic functions. Most people put
:   'method' documentation in the docstrings of their generic functions, but
:   if you set docstrings for individual methods then set this to nil.
: - =STYLE-SHEET= specifies the name of a /Cascading Style Sheet/ (.CSS) file
:   which will be used as the style for the document if you export it
:   to HTML from org mode."
:
: * Example
: ;;; (clod:document-package :mypkg "mypkg-doc.org"
: ;;;      :style-sheet "swiss.css" :title "My Package"
: ;;;      :author "F. B. Quux" :email "quux@gmail.com")
:
: * See also
: - [[document-packages]]
  • Function DOCUMENT-PACKAGES (packages file/stream &key (auto-links nil) (lines-between-sections t) (brief-methods t) (internal-symbols? t) (class-diagram nil) (style-sheet nil) (title *document-title*) (author *document-author*) (email *document-email*))
    * Arguments - PACKAGES :: A list of package objects, or symbols naming packages. - FILE/STREAM :: A string (filename), stream object, or =NIL=. Other arguments are the same as for [[document-package]]. * Returns A string, or nil. * Description Produces documentation for all the packages in =PACKAGES=, within a single file. See [[document-package]] for more details.
  • Function DOCUMENT-PACKAGE (pkg file/stream &key (auto-links nil) (lines-between-sections t) (brief-methods t) (internal-symbols? t) (class-diagram nil) (style-sheet nil) (title nil) (author *document-author*) (email *document-email*))
    * Arguments - PKG :: A package name or package object. - FILE/STREAM :: A string (filename), stream object, or =NIL=. - AUTO-LINKS :: Boolean. - LINES-BETWEEN-SECTIONS :: Boolean. - BRIEF-METHODS :: Boolean. - INTERNAL-SYMBOLS :: Boolean. - CLASS-DIAGRAM :: Boolean. - STYLE-SHEET :: A string. - TITLE :: A string. - AUTHOR :: A string. - EMAIL :: A string. * Returns A string, or nil. * Description Produce documentation for the package =PKG=. The documentation's destination depends on the value of =FILE/STREAM=: - =STRING=: documentation is written to the file named by the string. - =STREAM=: documentation is written to the already existing stream. - =NIL=: documentation is written to a string, which is then returned by this function. Explanation of optional arguments: - =TITLE=, =AUTHOR= and =EMAIL= specify the document title, the name of the author, and the email address of the author. - =STYLE-SHEET= specifies the name of a Cascading Style Sheet (.CSS) file which will be used as the style for the document if you export it to HTML from org mode. - If =AUTO-LINKS= is non-nil, *all* occurrences of symbol names within the text of docstrings will be interpreted as hyperlinks, regardless of whether they are marked up as hyperlinks. - If =LINES-BETWEEN-SECTIONS= is nil, do not output a horizontal line before each new section of documentation. - If =BRIEF-METHODS= is nil, document individual methods with their own sections, just like functions and generic functions. Most people put 'method' documentation in the docstrings of their generic functions, but if you set docstrings for individual methods then set this to nil. - If =INTERNAL-SYMBOLS?= is non-nil, document both internal and external (exported) symbols. If nil, only document external symbols. - If =CLASS-DIAGRAM= is non-nil, create a section after the toplevel package description, containing a description of the package hierarchy in the form of a GraphViz 'dot' diagram (see http://www.graphviz.org/). * Example ;;; (clod:document-package :mypkg "mypkg-doc.org" ;;; :style-sheet "swiss.css" :title "My Package" ;;; :author "F. B. Quux" :email "quux@gmail.com") * See also - [[document-packages]]