ScriptL v2

There are various contortions for getting your Common Lisp started in a shell script and running some of your code. In the end, you're still falling back on writing Lisp like you're writing anything else: write, run, stop, repeat.

If only we could do this:

(defun my-command (x)
  (format t "If this were a real command, we would use ~A responsibly~%" x))

;; Make a wrapper to call this in our running Lisp!
(make-command "my-command" 'my-command)

And then:

$ my-command "foo"
If this were a real command, we would use foo responsibly.
$

Well, of course, that's exactly what ScriptL lets you do. Since this calls your running lisp, it means you're live coding your commands.

Quick Guide

You can get ScriptL from Quicklisp via (ql:quickload "scriptl"). This will load scriptl and create the scriptl command for managing scripts (see below).

First however, you must start the server. It may be convenient to create a shell script that launches your favorite CL on login and does the following:

(scriptl:start)

The scripts in examples/ require unix-options for argument parsing as well, so to run them load that system as well:

(asdf:load-system :scriptl-examples)

Now you can call things via the generated shell scripts (you might have to call them via ./[command] from the local directory):

$ funcall princ '"hello world"'
hello world
$ eval '(princ "hello world")'
hello world
$ eval '(+ 1 1)'
2
$

Alternatively, once you have something you want as a more permanent command, you can use SCRIPTL:MAKE-COMMAND as above:

(make-command COMMAND-NAME FUNCTION-NAME &optional ERROR-HANDLER)

This will make a shell script called COMMAND, probably in your home directory (or wherever you started lisp). It will call FUNCTION-NAME, with any parameters passed to the script given as strings.

Alternatively, you can use the make-command script included, which is a ScriptL wrapper for itself:

$ make-script some-command some-function [error-function]

This will make the new script in the current working directory, because wrappered calls rebind *DEFAULT-PATHNAME-DEFAULTS* for your convenience. Amusingly, you can use make-script to generate itself:

$ make-script make-script scriptl:make-script scriptl:make-script-usage

This will overwrite itself with an updated copy, assuming you're using an OS which isn't picky about such things.

scriptl and registering scripts

After installing ScriptL, the scriptl command will be created in the bin/ directory. This is part of a small API for registering, listing, and creating scripts. For instance:

$ scriptl list
Script                           Description

SCRIPTL:SCRIPTL                - ScriptL management command: scriptl
$ scriptl make SCRIPTL:SCRIPTL
$ ls
scriptl
$

This is more useful with a lot of scripts loaded. For example, after loading my personal scripts:

$ scriptl list
Script                           Description

SCRIPTL.SCRIPT.JSON:JSON       - JSON manipulation commands: json, json2yaml,
                                 sexp2json, dir2json
SCRIPTL.SCRIPT.NEW:NEW         - Template system command: new
SCRIPTL.SCRIPT.YAML:YAML       - YAML manipulation commands: yaml2json
SCRIPTL:SCRIPTL                - ScriptL management command: scriptl
$

To register scripts for this list, you should make a function which calls MAKE-COMMAND for each script you wish to create, and then call SCRIPTL:REGISTER to register this:

(defun make-my-scripts ()
  (make-command ...)
  (make-command ...))

(scriptl:register 'my-scripts 'make-my-scripts "My script commands: foo, bar, baz")
(export 'my-scripts)

Now you would see the following:

$ scriptl list
Script                           Description

MY-PACKAGE:MY-SCRIPTS          - My script commands: foo, bar, baz
SCRIPTL:SCRIPTL                - ScriptL management command: scriptl
$

Defaults and the Header

A number of useful things are set up by default:

• The default package is COMMON-LISP-USER
• Wrappers set the current working directory
• The current script name is set in SCRIPTL:*SCRIPT*
• The parsed header is set in SCRIPTL:*HEADER*

As above, whenever a wrapper is run, ScriptL sets *DEFAULT-PATHNAME-DEFAULTS* for you, which is enough for most Lisp functions to find the right file.

However, for things like osicat, you may need to use (merge-pathnames FILENAME) to get the appropriate thing.

You may find it interesting to dispatch on the script name, which is set in SCRIPTL:*SCRIPT*. It's a pathname, so you can extract the various bits as appropriate.

Additionally, you can access the complete call information via SCRIPTL:*HEADER*, e.g.:

(defun show-header (&rest args) scriptl:*header*)

(You don't have to use &rest args; this is for demonstration.)

$ funcall show-header 1 2 3
#S(SCRIPTL:HEADER
   :VERSION 1
   :CWD #P"/path/to/current/dir/"
   :COMMAND (:FUNCALL 3 #P"./funcall")
   :FUN SHOW-HEADER
   :ARGS ("1" "2" "3")
   :ERROR-FUN NIL)
$

Most of this isn't particularly interesting or beyond what you already get, but it's there.

Readline

The v2 client adds simple support for the GNU readline library. There are two basic operations: readline PROMPT and addhistory LINE:

(defun test ()
  (format t "Type 'quit' to finish.~%")
  (loop as line = (readline "> ")
        until (string= line "quit") do
          (addhistory line)
          (format t "You typed: ~A~%" line)))

In the shell:

Type 'quit' to finish.
> foo
You typed: foo
> quit

If GNU readline support is compiled into the client (autodetected), this will have all the fancy editing features that provides, including history. If not, or if the function is not called from ScriptL, the prompt will be shown, but only basic terminal input is provided, and history is ignored.

getenv

Also included in v2 is support for remote getenv(3), allowing client-side configuration via environment variables. This is done via the scriptl:getenv function.

This will return the value of a variable on the remote side, or for functions not being called via a ScriptL command, will fall back to osicat-posix:getenv and return the environment for the running lisp.

In both cases, this returns the value of the variable as a string if set, or NIL.

I/O

The v2 protocol adds a custom client which supports standard lisp I/O operations on *standard-input* and *standard-output*. For example:

(defun test-read-line ()
  (format t "Read line: ~A~%" (read-line)))

$ echo foo | test-read-line
Read line: foo
$

Additionally, you can use read-sequence, read-byte, and read-char. The former two return octet vectors, which may be useful for processing data byte-by-byte.

Reading and writing may be interleaved as desired. Writing also supports write-sequence, write-byte, and write-char. In order to write a raw byte sequence, however, it must be specified as an array with :element-type '(unsigned-byte 8). Otherwise it will be treated as a general object, and be pretty-printed.

Results and Error Handling

When things run correctly, if there is output to *standard-output*, this is shown to the user. If there is no output, the return values are shown instead:

$ eval '(+ 2 2)'
4
$ eval '(values 1 2 3)'
(VALUES 1 2 3)
$

When things go wrong, the error condition is shown by default:

$ eval 'foo'
Error: UNBOUND-VARIABLE

The variable FOO is unbound.
$

If you want slightly more sophisticated handling, you can define an error handler when calling MAKE-SCRIPT, which will get passed the condition via HANDLER-BIND. If you handle it, return an non-NIL value. If you can't handle the condition, return NIL.

Examples

The examples/ directory has a number of examples; most of them are simply talking to the ScriptL server.

The example.lisp and test-cmd example shows how to make a more shell-scripty-feeling function with more conventional means.

Alternatively, if you look at src/make-script.lisp, you can see a (rather naïve) error handler which provides usage. MAKE-SCRIPT also handles its parameters appropriately if they're strings.

Details

This all works, of course, by making a server on a port in your Lisp and listening there. I really wanted to use Swank, but in the end the problem went from simply talking to swank to implementing READ in the shell, and that's a lot more work than just writing a new, more targeted server.

For V1, this relied on netcat. V2 presents a custom C client, which builds as part of the ASDF load operation, and supports a number of new features, such as I/O.

ScriptL initially used TCP port 4010. This is bad for a number of reasons .. each user can't have their own lisp, and anyone can access it. V2 uses Unix Domain Sockets by default, and only the owner can access it. This is restricted by file permissions.

You can still use the TCP server, by specifying (scriptl:start :internet). You can adjust the port by binding scriptl:*scriptl-port* to the desired port before doing this. This talks only to localhost. It's horribly insecure, though no moreso than swank/slime, really. If you're worried, or not the only person with access to your host, don't run it.

In theory, making this talk over an ssh tunnel would be pretty easy, but switching ports and doing the setup isn't at all nice.