cl-statsd

2017-01-24

StatsD client in Common Lisp

If you are not familiar with statsd read this. To learn about statsd metric types read this.

Speed

CL-STATSD is relatively fast:

(let ((statsd::*client* (statsd:make-sync-client)))
  (loop for i from 0 to 999999 do
    (statsd:counter "self-test" 1)))

Using this Docker image with defaults on busy VM.

Result

Result

Metrics API

  • counter (key value &key (rate) (client *client*))
  • inc (key &key (rate) (client *client*)) - shortcut for (counter key 1 ...)
  • dec (key &key (rate) (client *client*)) - shortcut for (counter key -1 ...)
  • timing (key value &key (rate) (client *client*))
  • with-timing ((key &optional (client '*client*)) &body body) - executes body and collects execution time
  • gauge (key value &key (rate) (client *client*))
  • set (key value &key (rate) (client *client*))

Sampling rate can be controlled using *random-range* parameter (default is 100). If set to 0 turns off sampling completely (equivalent of constant rate 1)

Pipelining

Given statsd:*client* is bound you can write something like this:

(statsd:pipeline ()
  (statsd:inc "qwe")
  (statsd:inc "ewq"))

Pipelining implemented using 'wrapper' client that gathers metrics, concatenates them and sends directly to the wrapped client transport. Currently CL-STATSD makes no effort to split pipelined data or otherwise respect/detect MTU.

Error handling

By default all errors simply ignored. You can customize this behaviour by providing :error-handler strategy:

(let ((statsd:*client* (statsd:make-sync-client :error-handler :throw)))
  (loop for i from 0 to 3 do
    (statsd:counter "example" (random 100))
    (statsd::transport.close (statsd::client-transport statsd:*client*))))
; Evaluation aborted on #<CL-STATSD:TRANSPORT-ERROR {1005445653}>.

New error handling strategies can be created by specializing handler-handle-error/2

Clients

null

Like /dev/null #### capture Queues metrics. Useful for debugging, testing

(let ((statsd::*client* (statsd:make-capture-client)))
  (statsd:counter "example" (random 100))
  (statsd:capture-client.recv))
"example:62|c"

sync

Calls transport synchronously #### async More like 'connection-as-a-service', runs in separate thread, all metrics queued first. To prevent queue from overgrowing async client understands throttling threshold (i.e. max queue length):

(let ((statsd:*client* (statsd:make-async-client :error-handler :throw
                                                 :throttle-threshold 5)))
  (loop for i from 0 to 999999 do
        (statsd:counter "example" (random 100))))
; Evaluation aborted on #<CL-STATSD:THROTTLE-THRESHOLD-REACHED {100414CEF3}>.

Throttle threshold can be set globally using *throttle-threshold* value or per async client (:throttle-threshold parameter).

Transports

CL-STATSD comes with the following transport: - usocket - sends meterics to statsd

New transports can be created by specializing - make-transport - transport.connect - transport.send - transport.close

TODO

  • [x] Async client throttling
  • [ ] Travis CI integration
  • [ ] Pipelines

License

MIT

Author
Ilya Khaprov <ilya.khaprov@publitechs.com>, Ilya Khaprov <ilya.khaprov@publitechs.com> and CONTRIBUTORS
Maintainer
Ilya Khaprov <ilya.khaprov@publitechs.com>
License
MIT
Categories
distributed