DYNAMIC-COLLECT =============== By Robert Smith DYNAMIC-COLLECT is a library for dynamic, continuable, and abortable collection of data. This code is useful for times when data needs to be collected at various points in a program, but it is inconvenient to modify the program and create new pipelines for passing data around. It is also useful when such pipelines would take away from the intent of the program. This code was originally created for tasks in code analysis, where warnings and errors were collected during the analysis, and then processed a posteriori. EXAMPLE ------- First, we will define a function which intends to do some kind of analysis of data, and warn if it's not the right type. In this case, we will check that it's an integer. If it's not, we'll warn that we wanted an integer. Actually, we will collect a message that represents the warning. (defun pass-1 (data) (unless (integerp data) (collect (format nil "Warning: Not an integer: ~S" data)))) Next, we will do a more stringent analysis. If the data is a non-null list, then we will require that the first element of the list needs to be a symbol. Perhaps this represents a function call. If the data indeed doesn't look like a function call, then we will collect an error, and specify that collection (and further analysis) must not continue, so we pass NIL to CONTINUEP. (defun pass-2 (data) (when (and (listp data) (plusp (length data))) (unless (symbolp (car data)) (collect (format nil "Error: A function call needs a symbol ~ in the first position, given: ~S" (car data)) :continuep nil)))) Now a simple function to process our collected messages. In this case, we will just print them out in a friendly fashion. (defun process-messages (messages) (if (null messages) (format t "No messages.~%") (format t "~@(~R~) message~:P:~%~{ >> ~A~%~}" (length messages) messages))) Finally, we have our main entry point to our analysis. We use WITH-DYNAMIC-COLLECTION, and do the passes on our data. We will log when a pass completes to the user. (defun main (data) (let ((messages (with-dynamic-collection () (pass-1 data) (format t ";;; Done with pass 1.~%") (force-output) (pass-2 data) (format t ";;; Done with pass 2.~%") (force-output)))) (process-messages messages))) Now for some test runs. First, we provide completely "legitimate" data (according to our passes). CL-USER> (main 5) ;;; Done with pass 1. ;;; Done with pass 2. No messages. NIL As seen, the passes complete, and we have no messages. Now let's do the analysis on something that warns on the first pass. CL-USER> (main :quux) ;;; Done with pass 1. ;;; Done with pass 2. One message: >> Warning: Not an integer: :QUUX NIL As seen, both passes complete, but we ended up with a warning we collected. Now let's do something that passes the second analysis, but warns on the first, again. CL-USER> (main '(hello)) ;;; Done with pass 1. ;;; Done with pass 2. One message: >> Warning: Not an integer: (HELLO) NIL Same thing. Finally, let's do something that will cause issues with both. CL-USER> (main '(5 hello)) ;;; Done with pass 1. Two messages: >> Warning: Not an integer: (5 HELLO) >> Error: A function call needs a symbol in the first position, given: 5 NIL Note this time that both warnings were collected and displayed. But more importantly, note that we never reached the end of the second pass; our collection aborted early. Despite this very contrived example, early termination is very useful. For example, if we are analyzing a file that ends up being unable to be parsed, we can collect an error message, and fail early. ENSURING CORRECTNESS -------------------- Using the variable *ENSURE-HANDLED-COLLECT*, we can error if the system detects that a COLLECT is used without a properly enclosing WITH-DYNAMIC-COLLECTION. By default, *ENSURE-HANDLED-COLLECT* is NIL, which means that unhandled COLLECT forms will just return their RETURN keyword parameter value. COMPOSING DYNAMIC COLLECTIONS ----------------------------- WITH-DYNAMIC-COLLECTION is actually composable via the notion of tags. A "tag" is an EQL-comparable thing (often a keyword) that lets one match up a WITH-DYNAMIC-COLLECTION with a COLLECT form. We simply provide tags to each, and collection will be matched up accordingly. For example: CL-USER> (format t "OUTER: ~S~%" (with-dynamic-collection (:tag :outer) (collect 1 :tag :outer) (format t "INNER: ~S~%" (with-dynamic-collection (:tag :inner) (collect 2 :tag :outer) (collect 3 :tag :inner))))) will print INNER: (3) OUTER: (1 2) as expected. By default, the tags are NIL (an EQL-comparable value), which is sufficient when there's no composition.