CL-SYSLOG

Common Lisp interface to local and remote Syslog facilities.

Features

• Local sysloging implemented via foreign-function calls.
• Remote syslog over UDP sockets.
• Optional, complete RFC5424 support.

Documentation

The source files are relatively well documented, so we recommend looking at them, but here's a quickstart guide.

Notice

• cl-syslog.udp has been removed due to duplicate functionality in cl-syslog: udp-log-writer
• All functionality has been moved into system cl-syslog package cl-syslog (Sorry if this causes breakage)

Basic Usage

To load cl-syslog, use Quicklisp or ASDF as usual:

> (ql:quickload :cl-syslog)
;; or, if ASDF can already see it:
> (asdf:load-system :cl-syslog)

Priorities and facilities are stated by their keyword name. For instance, the priority :warning on facility :user. All of these keywords are housed in the variables *priorities* and *variables*. Should you provide invalid facilities or priorities, the conditions cl-syslog:invalid-facility and cl-syslog:invalid-priority will be signaled.

To log to your local syslog daemon, use cl-syslog:log (note that this is shadowed, and is obviously not the same as cl:log!):

> (cl-syslog:log "MyApp" ':user ':warning "Low on memory.")

Then look in your /var/log/messages or other location if you have tweaked your /etc/syslog.conf. On macOS, you can use the Console application.

RFC 5424 Support

RFC 5424 is supported. (You can check if your CL-SYSLOG installation support it by checking if cl-syslog::rfc5424 is present in your *features*.)

The first thing you'll need to do is create an instance of the rfc5424-logger class, filling out as many details as you please. You're encouraged to fill out as many slots as you can so that log messages contain more information.

Of particular interest is the :log-writer initarg. This controls how log messages get processed in the end. The default log writer will use a function wrapping cl-syslog:log. Should you instead be interested in writing to standard output, you can supply (cl-syslog:stream-log-writer) as an argument, like so:

(defparameter *logger*
 (make-instance 'cl-syslog:rfc5424-logger
                ;; optional
                :facility ':local0
                :app-name "MyApp"
                :hostname "computer.local"
                :log-writer (cl-syslog:stream-log-writer)))

There are a few options for log writers, including a UDP log writer for remote RFC-compliant logging. You can always support your own lambda function, too.

Once you have a logger, the simplest way to create a compliant message is to use cl-syslog:format-log:

> (cl-syslog:format-log *logger* ':warning "Low on memory.")
<132>1 2018-12-13T22:17:56Z computer.local MyApp 69840 - - Low on memory.

RFC 5424 has support for sending structured data within the log message. It's a somewhat complicated arrangement, in that the structured data has to be defined and "agreed upon". Specifically, there are a set of IETF-approved structured data. They're identified by "structured data IDs", and the standard IDs are timeQuality, origin, and meta. Each of these have different structured data parameters.

Personal structured data IDs must have some name, followed by @, followed by a usually 4 or 5 digit integer. For instance, mycompany@0001 is a valid personal structured data ID. One can define a structured data ID using cl-syslog:define-structured-data-id. The definition of the standard origin ID is:

(define-structured-data-id |origin| (:standard t)
  (|ip| :allow-repetitions t
        :validator 'ip-address-p)
  |enterpriseId|
  (|software| :length (integer 0 40))
  (|swVersion| :length (integer 0 32)))

(This can be seen in the file rfc5424-reserved.lisp.)

Your own structured data ID might be:

(cl-syslog:define-structured-data-id |mycompany@0001| ()
  |who|
  |what|)

Note that the field names are escaped to follow the RFC's usual idiom. Also node that the field names are here symbols scoped to your package. CL-SYSLOG exports all of the symbols of the standard ID's.

How can we use these? We use the log macro rfc-log. It's a macro because it does extra work at compile time to build code that only runs if the log message is to be sent.

Suppose we've defined the above structured data ID. Then we might log the following:

> (cl-syslog:rfc-log (*logger* :warning "Running out of memory! I got ~D byte~:P left!" 10)
    (cl-syslog:|origin|
     cl-syslog:|ip| "1.2.3.4"
     cl-syslog:|software| "My Testing App"
     cl-syslog:|swVersion| "12.2")
    (|mycompany@0001|
     |who| "John Doe"
     |what| "Needs to buy a new computer."))

;; output:
<132>1 2018-12-13T22:28:02Z computer.local MyApp 69840 - [origin ip="1.2.3.4" software="My Testing App" swVersion="12.2"][mycompany@0001 who="John Doe" what="Needs to buy a new computer."] Running out of memory! I got 10 bytes left!

Optionally, a message ID string can be supplied. This is useful if the message is a part of a category of similar messages. Suppose the message ID is "LOG1234", then we could write the above message as:

> (cl-syslog:rfc-log (*logger* :warning "Running out of memory! I got ~D byte~:P left!" 10)
    (:msgid "LOG1234")
    (cl-syslog:|origin|
     cl-syslog:|ip| "1.2.3.4"
     cl-syslog:|software| "My Testing App"
     cl-syslog:|swVersion| "12.2")
    (|mycompany@0001|
     |who| "John Doe"
     |what| "Needs to buy a new computer."))

;; output:
<132>1 2018-12-13T22:29:36Z computer.local MyApp 69840 LOG1234 [origin ip="1.2.3.4" software="My Testing App" swVersion="12.2"][mycompany@0001 who="John Doe" what="Needs to buy a new computer."] Running out of memory! I got 10 bytes left!

RFC 5424 Caveats

Currently, Unicode isn't properly handled. It is expected all data is ASCII. It is also expect that your Lisp's char-code matches ASCII. The necessary BOM will not be inserted for Unicode.

The RFC 5424 log handling functions generally write to streams, but the entry points cons up strings (if and only if the log message is actually sent). There is currently no support for allowing streaming log data at the log generation level.

Many log processors do not implement all of RFC 5424, but nonetheless cope rather well with the format, and can understand the RFC's key=value pairs.

Raw C Functions

The raw C interfaces are CFFI-accessible by their standard UNIX names: openlog, closelog, and syslog. None of these are necessary unless you are looking for complete control. Beware, if you mess these calls up, you will break your Lisp process.