Validation and conversion between user and internal data.

cl-data-format-validation

.. -*-RST-*-
.. |VERSION| replace:: 0.1.6

========================
DATA FORMAT VALIDATION
========================

:Author:       Dr John A.R. Williams
:Contact:      J.A.R.Williams@jarw.org.uk
:date:         2010/04/08
:status:       Initial Public Release
:version:      |VERSION|
:copyright:    © 2011 J.A.R. Williams
:abstract:     DATA-FORMAT-VALIDATION is a library for Common Lisp providing a
     consistent regular interface for converting (and validating) external data
     (in the form of strings usually) into internal data types and
     for formatting internal data back into external presentable
     strings, all according to a conversion or type specification. 

.. meta::
   :keywords: Common Lisp

.. contents:: Table of Contents

.. |DFV| replace:: DATA-FORMAT-VALIDATION
.. |JARW| replace:: John A.R. Williams

Download and Installation
=========================

|DFV| together with this documentation can be downloaded from the git
repository at
`<git://github.com/willijar/cl-data-format-validation.git>` or from
<http://www.jarw.org.uk/lisp/cl-data-format-validation.tar.gz>. The
current release version is |VERSION|.

|DFV| comes with a system definition for 
`ASDF <http://www.cliki.net/asdf>`_ and is compiled and loaded in the usual
way. It depends upon `CL-PPCRE <http://weitz.de/cl-ppcre/>`_.

|DFV|  is made available under the terms of the GPL v3 license - see
the file ``LICENSE.txt`` for details.

Support
=======

For questions, bug reports, feature requests, improvements, or patches
please email <J.A.R.Williams@jarw.org.uk>.

The API
=======

generic function **parse-input** `specification value &key &allow-other-keys => object`
   Validate and parse user input according to
   specification, returning the validated object. Throws an invalid-input
   condition if input is invalid.  If specification is a list the first
   element specifies the actual validation method and the rest of the
   list are passed as keyword arguments to the specific method::

     (parse-input '(integer :min 0) input)

   will return the integer value from strin if it is >0, or signal and
   invalid-input error if not and::

     (parse-input '(member :type integer :set (1 5 7)) input)

   will return it only if it has a value in the set.

   The `use-value` restart may be used to provide substitute value if the
   input is invalid.

generic function **format-output** `specification value &key &allow-other-keys => string`
   Return a string representation of value formatted
   according to a specification. If specification is a list the first
   element specifies the actual validation method and the rest of the
   list are passed as keyword arguments to the specific method e.g.::

    (format-output '(date :fmt :rfc2822) (get-universal-time))
    >"Mon, 10 Jul 2006 15:43:45 +00"

generic function **equivalent** `specification input reference &key &allow-other-keys => boolean`
  Return true if the input and reference values can be consider
  equivalent according to the specification. The default is to test
  using **equal**.

generic function **parse-options** `spec options-list &optional allow-other-options => options`
  Parse an option list (alist of names and strings to be parsed)
  against a specification. The specification is a list of entries each
  of which lists the name, and optionally the type specification (to
  be used by **parse-input**) and the default value to be used if there
  is no entry in the options-list. The
  output is an alist of names and the parsed or default values. Options in
  `options-list` not in spec are not returned and will signal a correctable
  `unknown-option` error unless `allow-other-options` is true.

generic function **parse-arguments** `spec argument-string &optional allow-spaces => arguments`
  Parse a string of whitespace delimited arguments according to spec.
  The specification is a list of entries each
  of which lists the name, and optionally the type  specification (to
  be used by **parse-input**) and default values. The
  output is an alist of variable names and parsed values.
  If allow-spaces is true, last element can contain spaces
  (i.e. trailing spaces are not trimmed).

formatter function **eng** `os arg &optional colon-p at-p d padchar exponentchar`
  Formatter which outputs its numerical argument `arg` in engineering format
  to stream `os`.
  It takes arguments `d,padchar,exponentchar` where
  `d` is the number of decimal places to display after the decimal point
  `padchar` is the character to pad the start of the number
  `exponentchar` is the character to use to display between radix and exponent
  It also takes the : modifier which will cause it to output the exponent
  as an SI units prefix rather than a number.

  e.g. `(format nil \"~/eng/\" 35000) => \"35.00e+3\"`

formatter function **date** `os utime &optional colon-p at-p precision 6 timezone`
  Formatter which formats a universal time for output as a date and time

  Modifiers:

  - os: an output stream designator
  - arg: a universal time
  - colon-p: a generalised boolean (default false).
             If true use month and day names in date
  - at-p: a generalised boolean (default false) - if true print in yyyy-mm-dd
          (sortable) format rather than dd-mm-yyy
  - precision: what precision to print it to. 6 is to the second,
             7 includes timezone, a negative number counts backward.
  - timezone: an integer (default `*timezone*`).
            If nil no timezone used and time is in current timezone
            adjusted for daylight saving time.
        
  e.g. `(format nil \"~/date/\" (get-universal-time)) => \"19-03-2009 08:30\""`

function **join-strings** `strings &optional (separator #\space) => string`
  Return a new string by joining together the list of  `strings`,
  separating each string with a `separator` character or string

function **split-string** `string &key count delimiter remove-empty-subseqs => list`
  Split `string` along whitespace as defined by the sequence `delimiter`.
  Whitespace which causes a split is elided from the result.  The whole
  string will be split, unless `max` is provided, in which case the
  string will be split into this number of tokens at most, the last one
  containing the whole rest of the given `string`. If
  `remove-empty-subseqs` is true zero length entries are removed. This
  is similar to `split-sequence` however it only takes a string input and
  the delimiter may be a string.

Type Specifications
===================

A type specification is an S-expression composed of a symbol
specifying the particular conversion and a keyword argument list of
qualifiers. Specific methods of **parse-input** and **format-output**
are specialised on the conversion type symbol and take the remainder
of the S-expression as an argument list. Adding your own conversions
is simply a matter of providing appropriately specialised
methods. The intended semantics are that the if the output from
**format-output** is read back in using **parse-input** with thye same
type specifications then an equivalent object should result.

Many conversions take the `nil-allowed` argument which
convert an empty or all whitespace string to nil corresponding to a
null input, otherwise an empty string is considered invalid input.
Methods specialisations are provided for the following types:


**boolean** `&key`
  Converts typical user boolean values (e.g. "TRUE", "Y",  "0") into a
  boolean type. On output "TRUE" and "FALSE" are used.

**bit-vector** `&key`
  Converts between a string of 0 and 1s and a bit vector.

**date** `&key nil-allowed zone fmt` Uses the `parse-time` library of
  Jim Healy and Daniel Barlow to convert to internal universal time in
  specified timezone `zone` which to defaults to special variable
  `*timezone*` for output but to `nil` for parsing input. If `zone` is
  nil the time will be in the current timezone allowing for local
  daylight savings time - otherwise it is in the specified timezone,
  which will be written out.

  `fmt` is a keyword specifying the output format to be used as
  follows.

   A stand alone formatter of the same name is also provided.
  
  :RFC2822   - output as per RFC2822 for internet messages
  :SHORT     - output in a shorter format (same as :ISO)
  :TIME-ONLY - outputs time as hh:mm:ss
  :DATE-ONLY - outputs date as dd-mm-yyyy
  :ISO       - output as per ISO 8602 (default)

**dimensional-parameter** `&key padchar decimal-places tol`
  Converts between a string which includes units and normal scaling
  suffixes and a cons of the numerical value and the base units
  string. `padchar` and `decimal-places` are as per **eng**.

  A dimensional comparator is equivalent if the numerical values and
  the units are equivalent.
 
**eng** `&key units padchar decimal-places`
  Parse a number suffix
  with units. The standard engineering prefixes are assumed for the
  units (but with 'u' instead of 'µ'). The appropriatly scaled
  floating point value is returned and if the `units`. If `units` is a
  string then the input units suffix must match. On output the number
  will be scaled and the appropriate engineering prefix used.
  A general purpose formatter of the same name is also provided.

**filename** `&key if-invalid replacement`
  Return a safe filename from a string path value.
  May return an error or replace invalid characters with the specified
  replacement letter (default '-');

**headers** `&key stream skip-blanks-p field-specifications
  preserve-newlines-p termination-test if-no-specification`
  Parse or format internet message style headers. `parse-input` takes
  either a string or stream as the input value. 

  `field-specifications`
  is either an a-list by field name of giving the parse type
  specification to be applied recursively for that field or a function
  which returns the parse type specification and a `present-p` values
  in the usual way. `if-no-specification` specifies either a type
  specification to be used if the field is not found in
  `field-specifications`, `:error` for this case to be flagged as an
  error or `:ignore` to ignore fields without specifications.
  If defaults to `nil` i.e. value is passed through as a string
  without parsing.

  `skip-blanks-p` will allow the parser to skip leading blank lines on
  the input. `termination-test` is a test function which of one
  argument (a string - a line) which should return true if the
  argument terminates the headers - default tests for a zero length
  line. If `preserve-newlines-p` is true then continuation lines will
  keep their newline characters, otherwise the newlines and first
  continuation character are removed.

  `format-output` will write its output to `stream` if it is given,
  otherwise it will return a string containing the output headers.   

**integer** `&key min max nil-allowed radix format`
  Converts to an integer between `min` and `max` (inclusive, and if
  specified). `radix` specified the base (in the usual way). `format`
  specifies the format control string to be used for output.

**list** `&key separator type min-length max-length`
  Return a list of objects delimited by the given `separator`
  string. Each member is recursively checked the nested type
  (another type specification). If specified `min-length` and
  `max-length` specify the required length bounds. The type
  specification may be a list of type specifications applied to each
  element in turn or a single type specification applied to all
  elements (note there is an ambiguity if you specify a list of one
  symbol - in this it is taken as a conversion for the first element only).

**member** `&key type set test  key`
  Recursively uses `type` to convert string to internal object which
  is then checked for membership of the list `set` using `key` and
  `test`(default is equal allowing for string tests).

**nil** `&key`
  Return string unchanged.

**number** `&key min max nil-allowed format radix tol`
  Converts to a general number between `min` and `max` (inclusive, and if
  specified). `radix`
  specified the base (in the usual way). `format` specifies the format
  control string to be used for output. The `parse-number` library of
  Matthew Danish is used to do the conversion.

  `tol` is the tolerance to be used for **equivalence** testing - it
  can either be a multiplier applied to the reference value or a
  function of two arguments - the input and the reference value.

**pathname** `&key must-exist wild-allowed nil-allowed`
  Convert input to a pathname. If `wild-allowed` is true then the
  pathname is allowed to be wild, otherwise if `must-exist` is true
  then the pathname must correspond to an existing file (checked using
  probe-file.  

**pathnames** `&key must-exist wild-allowed nil-allowed`
  Return a list of pathnames delimited by ':', each checked as for **pathname**

**read** `&key multiplep type package`
  Uses the lisp reader with the current package set to
  `package`. `type` is a Common Lisp type against which the read
  object(s) is checked. If `multiplep` is true then read will be
  continually called until all characters are used up and the results
  are returned as a list. On output, if `multiplep` is true list of
  objects are separated by a space and written readably. 

**roman**
  Convert between roman numerals (up to 4000) and an integer

**string** `&key strip-return nil-allowed min-word-count max-word-count min-length max-length`
  Validates that the string is between `min-length` and `max-length`
  characters long (inclusive, and if specified) and the word count is
  between `min-word-count` and `max-word-count`. 
  Whitespace is trimmed from the returned string, and if
  `strip-return` is specified the RETURN characters are stripped from
  the string (useful when handling input from http forms).

**symbol** `&key nil-allowed package convert`
  Returns a symbol from the string interned into `package` (default
  is the keyword package). `conversion` is a function applied to the
  string before it is interned (default identity) which may for
  example be used to change case or map special characters.

**time-period** `&key`
  A time period in hours, minutes and (optionally) seconds is
  converted into an integer number of seconds. ':' is used as the
  delimiter between fields.

Conditions and Restarts
=======================

**invalid-format** 
  is signalled if the input doesn't meet the type specification. It has
  readers `invalid-format-value` and `invalid-format-reason`.

**use-value** 
  restart may be invoked to specify a result to be used if
  invalid-input is signalled.

**use-default**
  This restart is available for **parse-options** and
  **parse-arguments** and will result in a default specified value
  being used.


Acknowledgements
================

Matthew Danish for the parse-number library used and enclosed with
this.

Daniel Barlow and Jim Healey for the parse-time library.



cl-data-format-validation

Upstream URL

Author

Maintainer

License

data-format-validation

Version:

License:

Authors:

Maintainers:

Dependencies:

Dependencies (1)

Dependents (1)