cl-messagepack
2020-10-16
A Common-Lisp implementation of Message Pack serialization.
-- markdown --
CL-MESSAGEPACK
A Common Lisp implementation of the MessagePack (http://msgpack.org/) serialization/deserialization format, implemented according to http://wiki.msgpack.org/display/MSGPACK/Format+specification.
Depends on flexi-streams
and babel
. Floating point values are
supported only on SBCL currently.
Status: first draft encoder and decoder implemented, added extensions for some Lisp data types (see below), simple tests.
Extensions to the Message Pack specification
(C0) 'NIL'
This translates to NIL
in Lisp directly, but see C2
(False
) below, too.
(C2) 'False'
On encoding this can be achieved via :false
; when this is encountered
during decoding, NIL
is returned to Lisp as long *use-false*
is kept NIL
.
Earlier implementation of CL-MESSAGEPACK
The previous version used the bytes #xC4
, #xC5
, #xC6
, and #xC7
to encode
a 'pure' cons (a cons whose CDR
is not a cons),
symbols, symbols as a number (via a lookup table), and rationals.
As these encodings are no longer allowed by the MSGPACK specification this
functionality has been removed; you can use *EXTENDED-TYPES*
to achieve similar things, though.
Extended Types
MSGPACK allows for a range of "Extended Types"; these consist of one of the
bytes #xC7
to #xC9
resp. #xD4
to #xD8
, a one-byte type number, and an
array of bytes for the data (which can optionally be interpreted as an integer ID).
A simple use case is eg. to identify pieces of data across a messagepack-RPC channel (like eg. http://github.com/neovim/neovim does):
(defparameter *my-type-list*
(messagepack:define-extension-types
'(:numeric
0
Buffer
Window
Tabpage
...)))
(defparameter *my-lookup-table*
(make-array 10 :adjustable t :initial-element nil))
(let ((messagepack:*extended-types* *my-type-list*))
(messagepack:*lookup-table* *my-lookup-table*)
(messagepack:decode-stream stream))
Now receiving an reply with an item of extended type 0 will
automatically build an instance of the class BUFFER
, and the ID
slot will
be filled with the received ID, so that passing that instance to another
query can be converted into a matching messagepack extended type element.
This provides type-safe communication across this RPC link.
The classes don't have to be defined ahead of time; the call to DEFINE-EXTENSION-TYPES will create their definition if needed.
Please remember that only the id gets transmitted; if you want to get the
same object (with same as in EQ
), you'll need to make sure that
the correct object is looked up again; this is what *LOOKUP-TABLE*
above is for. Remember to bind that per RPC-connection to avoid duplicate IDs,
and to invalidate it if the remote process changes!
For more advanced usage CL-MESSAGEPACK
provides a base class EXTENSION-TYPE
that can be used to define classes with more slots:
(defclass type1 (cl-messagepack:extension-type)
( slots... ))
Please note that encoding is currently limited to the #xC7
byte, and
therefore imposes a 255 byte limit for the byte array.
Testing
Copy the cl-messagepack
directory to the local-projects
directory
of your Quicklisp install, then
(ql:quickload :cl-messagepack)
(asdf:test-system :cl-messagepack)
in a REPL (tested under SBCL and CCL under Linux x64).