lzip (LZMA) (de)compression using bindings to lzlib

cl-lzlib

#+TITLE: cl-lzlib
#+AUTHOR: Guillaume LE VAILLANT
#+DATE: 2022-06-20
#+EMAIL: glv@posteo.net
#+LANGUAGE: en
#+OPTIONS: num:nil toc:nil html-postamble:nil html-scripts:nil
#+HTML_DOCTYPE: html5

* Description

*cl-lzlib* is a Common Lisp library for lzip (LZMA) compression/decompression
using bindings to the lzlib C library.

* License

*cl-lzlib* is released under the GPL-3 license or later. See the [[file:LICENSE][LICENSE]] file
for details.

* Dependencies

*cl-lzlib* requires:
 - [[https://cffi.common-lisp.dev/][cffi]]
 - [[https://codeberg.org/glv/cl-octet-streams][cl-octet-streams]]
 - [[https://lparallel.org/][lparallel]]
 - [[https://www.nongnu.org/lzip/lzlib.html][lzlib]]
 - [[https://trivial-gray-streams.common-lisp.dev/][trivial-gray-streams]]

*cl-lzlib* can be easily installed with [[https://www.quicklisp.org][quicklisp]].

A *lzlib* package should be available in almost every GNU/Linux or *BSD
distribution. For example it is called /lzlib/ on Gentoo and Guix, and
/liblz1/ on Debian and Ubuntu.

* API

The library can be loaded with the usual:

#+BEGIN_SRC lisp
(asdf:load-system "lzlib")
#+END_SRC

or

#+BEGIN_SRC lisp
(quicklisp:quickload "lzlib")
#+END_SRC

The functions will then be available in the ~lzlib~ package.

** Compression

The compression can be customized using serveral parameters:
 - The maximum size of members inside the lzip archive can be indicated with the
   /member-size/ parameter. The default value is 2 PiB.
 - The compression level can be indicated either using the /level/ parameter, or
   by using both the /dictionary-size/ and /match-len-limit/ parameters. The
   default level is 6, which sets /dictionary-size/ to 8 MiB and
   /match-len-limit/ to 36. /level/ must be between 0 (fast encoder) and
   9 (strong compression).
 - When the /threads/ parameter is greater than 1, the data is read by blocks
   of /block-size/ bytes and the blocks are compressed in parallel. The default
   block size is the double of the dictionary size.

#+BEGIN_SRC lisp
(compress-stream input output &key threads level member-size block-size dictionary-size match-len-limit) => t
#+END_SRC

Read the data from the /input/ octet stream, compress it, and write the result
to the /output/ octet stream.

#+BEGIN_SRC lisp
(compress-file input output &key threads level member-size block-size dictionary-size match-len-limit) => t
#+END_SRC

Read the data from the /input/ file, compress it, and write the result to the
/output/ file.

#+BEGIN_SRC lisp
(compress-buffer buffer &key start end threads level member-size block-size dictionary-size match-len-limit) => bytes
#+END_SRC

Read the data between the /start/ and /end/ offsets in the /buffer/, compress
it, and return the resulting octet vector.

#+BEGIN_SRC lisp
(make-compressing-stream output-stream &key level member-size dictionary-size match-len-limit) => stream
#+END_SRC

Return a stream that will compress the bytes written to it at the given
compression /level/ and write them to the /output-stream/.

#+BEGIN_SRC lisp
(with-compressing-stream (stream output-stream &key level member-size dictionary-size match-len-limit) &body body)
#+END_SRC

Within /body/, /stream/ is bound to a compressing stream for the given
compression /level/ and /output-stream/. The result of the last form of /body/
is returned.

** Decompression

The decompression can be customized using serveral parameters:
 - When an archive has some trailing data, if /ignore-trailing/ is ~nil~ an
   error will be generated, otherwise the trailing data will be ignored.
 - When an archive has some trailing data looking like a corrupt header, if
   /loose-trailing/ is ~nil~ an error will be generated, otherwise the trailing
   data will be ignored.
 - When the /threads/ parameter is greater than 1, the library will decompress
   the members of the archive in parallel. Note that if the archive only has
   1 member, the decompression will be faster and use less memory when
   /threads/ is also 1.

#+BEGIN_SRC lisp
(decompress-stream input output &key threads ignore-trailing loose-trailing) => t
#+END_SRC

Read the data from the /input/ octet stream, decompress it, and write the result
to the /output/ octet stream.

#+BEGIN_SRC lisp
(decompress-file input output &key threads ignore-trailing loose-trailing) => t
#+END_SRC

Read the data from the /input/ file, decompress it, and write the result to the
/output/ file.

#+BEGIN_SRC lisp
(decompress-buffer buffer &key start end threads ignore-trailing loose-trailing) => bytes
#+END_SRC

Read the data between the /start/ and /end/ offsets in the /buffer/, decompress
it, and return the resulting octet vector.

#+BEGIN_SRC lisp
(make-decompressing-stream input-stream &key ignore-trailing loose-trailing) => stream
#+END_SRC

Return a stream that will supply the bytes resulting from the decompression of
the data read from the /input-stream/.

#+BEGIN_SRC lisp
(with-decompressing-stream (stream input-stream &key ignore-trailing loose-trailing) &body body)
#+END_SRC

Within /body/, /stream/ is bound to a decompressing stream for the given
/input-stream/. The result of the last form of /body/ is returned.

* Tests

The tests require the [[https://common-lisp.net/project/fiveam/][fiveam]] package. They can be run with:

#+BEGIN_SRC lisp
(asdf:test-system "lzlib")
#+END_SRC


cl-lzlib

Upstream URL

Author

License

lzlib

Version:

License:

Authors:

Dependencies:

lzlib-tests

Version:

License:

Authors:

Dependencies:

Dependencies (6)

Dependents (0)