About lQuery

lQuery is a DOM manipulation library written in Common Lisp, inspired by and based on the jQuery syntax and functions. It uses Plump and CLSS as DOM and selector engines. The main idea behind lQuery is to provide a simple interface for crawling and modifying HTML sites, as well as to allow for an alternative approach to templating.

How To

Load lQuery with ASDF or Quicklisp.

(ql:quickload :lquery)

First, lQuery needs to be initialized with a document to work on:

(defvar *doc* (lquery:$ (initialize (asdf:system-relative-pathname :lquery "test.html"))))

After that, you can use the $ macro to select and manipulate the DOM:

(lquery:$ *doc* "article")
(lquery:$ *doc* 
  "article"
  (add-class "fancy")
  (attr "foo" "bar"))
  

To render the HTML to a string use SERIALIZE. If you want to save it to a file directly, there's also WRITE-TO-FILE.

(lquery:$ *doc* (serialize))
(lquery:$ *doc* (write-to-file #p"~/plump-test.html"))

So a quick file manipulation could look something like this:

(lquery:$ (initialize (asdf:system-relative-pathname :lquery "test.html"))
  "article"
  (append-to "#target")
  (add-class "foo")
  (root)
  (write-to-file #p"~/plump-test.html"))

Aside from using selectors as the first step, it's also possible to use any other variable or list and operate on it.
Since 2.0: Literal function calls need to be added with INLINE. Note that the result of the inline function will be used as if literally put in place. For example, an inlined function that evaluates to a string will result in a CSS-select.

(lquery:$ (inline (list node-a node-b))
  "article"
  (serialize))

Selectors can come at any point in the sequence of lQuery operations and will always act on the current set of elements. If an operation evaluates to a list, array, vector or a single node, the current set of elements is set to this result.

(lquery:$ *doc*
  "a"
  (text "Link")
  (inline (lquery:$ *doc* "p"))
  (text "Paragraph"))

This is equivalent to the following:

(lquery:$ *doc*
  "a" (text "Link"))
(lquery:$ *doc*
  "p" (text "Paragraph"))

Functions in the argument list will be translated to a function invocation with the current list of elements as their argument.

(lquery:$ *doc*
  "a" #'(lambda (els) (aref els 0)))

lQuery^2.0 also supports compile-time evaluation of forms, whose results are then put in place of their function calls:

(lquery:$ *doc* (eval (format NIL "~a" *selector*)))

Keep in mind that the lexical environment is not the same at compile-time as at run-time.

Often times you'll also want to retrieve multiple, different values from your current set of nodes. To make this more convenient, you can use the COMBINE form:

(lquery:$ *doc*
  "a"
  (combine (attr :href) (text))
  (map-apply #'(lambda (url text) (format T "[~a](~a)" text url))))

lQuery uses vectors internally to modify and handle sets of nodes. These vectors are usually modified instead of copied to avoid unnecessary resource allocation. This however also means that lQuery functions are possibly side-effecting. If you pass an adjustable vector into lQuery through INLINE or similar, it will not be copied and therefore side-effects might occur. lQuery will automatically copy everything else that isn't an adjustable vector through ENSURE-PROPER-VECTOR. If you do want to pass in an adjustable vector, but make sure it doesn't affect it, use COPY-PROPER-VECTOR.

Test Suite

To ensure that functions are at least somewhat stable in their behaviour, lQuery includes a test suite. You can load this through Quicklisp/ASDF with

(ql:quickload :lquery-test)
(lquery-test:run)

The tests are rather loose, but should cover all functions to at least behave mostly according to expectation.

Extending lQuery^3.1

lQuery allows extension in a couple of ways. The most important of which are node functions themselves, which come in two flavours: lquery-funs and lquery-list-funs. Any lquery function resides in the package LQUERY-FUNCS, which is automatically scanned by the $ macro. The two macros responsible for defining new lquery functions automatically place the resulting operations in this package for you.

(define-lquery-function name (node-name &rest arguments) &body body)
(define-lquery-list-function name (vector-name &rest arguments) &body body)

Any function generated by these macros can be called either with a single node or a vector of nodes. In the case of a regular node operation, if it receives a vector of nodes, the function is called once for each node and the results are collected into a vector, which is then returned. If it receives a single node, only a single result is returned. In the case of a node list function, the return value can be either a vector or a single value, depending on what the goal of the operation is. It is expected that node list functions will modify the given vector directly in order to avoid unnecessary copying.

Some constructs would be very cumbersome to write as functions, or would simply be more suited in the form of a macro. To allow for this, lQuery^3.1 includes a mechanism of $ local macros. The previously mentioned forms like INLINE are handled through this system. Just like lquery functions, lquery macros reside in their own package LQUERY-MACROS. The responsible macro for defining new lquery macros will automatically place it in there for you.

(define-lquery-macro name (previous-form &rest arguments) &body body)

The $ macro itself can be extended as well by providing additional argument- or value-handlers. The following two macros make this possible:

(define-argument-handler type (argument-name operator-name) &body body)

Argument handlers transform the arguments at compile-time. For example, this would allow an extension to turn literal arrays into lists so they can be processed as well.

(define-value-handler type (variable-name operator-name) &body body)

Value handlers on the other hand determine the action at run-time. This is mostly useful for defining special actions on certain variable values.

What's New

3.1.0

Renamed DEFINE-NODE-\* macros into more sensible DEFINE-LQUERY-*. Added macro system, new standard COMBINE macro.

3.0.0

Complete rewrite of everything. This version is compatibility breaking. While the node functions themselves perform just the same as before (with one or two exceptions), lQuery now uses vectors instead of lists internally. If you ever relied on lQuery return values, this version will most likely break your code. Effort has been made to keep upgrading as simple as possible though; passing lists into an lQuery chain automatically transforms it for example.

Thanks to the change to Plump, lQuery is now also able to parse almost any kind of X/HT/ML document, which was not well possible previously. And thanks to switching to CLSS, lQuery is now much faster at selecting nodes from the DOM.

2.0.0

Added extension system and INLINE, EVAL handling. Revamped base macros to be more stable and simple.

Further Reading

• lQuery's symbol index.
• Plump, the HTML parser and DOM library lQuery is based on.
• CLSS, the CSS-selector DOM traversal engine.
• Clip, a templating system based on lQuery.

Support

If you'd like to support the continued development of Trial, please consider becoming a backer on Patreon: