http2

#+TITLE: http/2 in Common Lisp

This is a (work in progress) implementation of HTTP/2 protocol as described in
RFC9113 (and RFC7540 before) and RFC7541 (HPACK).

The core part of the library is implementing writing http2 frames, reading them,
and reacting on read frames. What to do with the frames is managed by classes
~CONNECTION~ and ~HTTP2-STREAM~ that are supposed to be extended by application to
do something reasonable with headers and payload; see [[./client/client.lisp][client.lisp]] or [[./server/server.lisp][server.lisp]] for an
example.

** Status and quickstart
Almost all parts standards implemented, see below for exceptions.

The library definitely allow to implement a client or server, and a simple ones are attached to it as examples.

*** Client

The bundled client requests remote page and returns values of body (converted to string), result status, and response headers.

Improvements to allow for cookie management, decoding to proper charset, .. are out of scope of this library, as they would probably be better taken over from some existing http1 library or incorporated there.

Improvements to provide stream for reading the output, to query several resources on one server, and to receive pushes are considered for future, as they are http2 specific.

#+begin_src lisp
  (ql:quickload 'http2/client)
  (http2/client:retrieve-url "https://example.com")
#+end_src

#+RESULTS:
#+begin_src text
<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" conten...[sly-elided string of length 1256]"
"200"
(("content-length" . "1256") ("x-cache" . "HIT") ("vary" . "Accept-Encoding")
 ("server" . "ECS (bsa/EB24)")
 ("last-modified" . "Thu, 17 Oct 2019 07:18:26 GMT")
 ("expires" . "Thu, 16 Jun 2022 09:35:21 GMT")
 ("etag" . "\"3147526947+ident\"") ("date" . "Thu, 09 Jun 2022 09:35:21 GMT")
 ("content-type" . "text/html; charset=UTF-8")
 ("cache-control" . "max-age=604800") ("age" . "239205"))
#+end_src

*** Server
Example https server (that signals h2 support with ALPN) is implemented; TLS key and certificate is needed ([[./scaffolding/run-server.lisp][test script]] generates it when missing). It handles requests one by one by default; this is trivial to improve in several ways (thread for request, workers, polling...) with customizing ~*dispatch-fn*~  but out of scope of this project.

#+begin_src lisp
  (ql:quickload 'http2/server)
  (http2::create-server 1234 "key.pem" "cert.pem")
#+end_src

Then point your browser (not eww nor drakma, of course - must support http2) to https://localhost:1234 or
#+begin_src sh
  [zellerin@winter http2]$ curl http://localhost:1234/
  Hello World
#+end_src

Customize PEER-ENDS-HTTP-STREAM to send something else (or process-end-headers if you do not want to see their data), and maybe add a method to ADD-HEADER to pay attention to what other headers than URL the client sends.

For simple cases, handlers can be defined to send data as text or binary streams, see sample server. Splitting the payload based on window size is done.

** Missing pieces
- No handling of priorities is implemented. This is OK, as these are only
  suggestions, and they are more or less dropped in RFC9113 anyway.
- Push promises are not implemented in the client. This is OK, they are disabled
  by default (settings)
- Option to encode headers to /Literal Header Field Never Indexed/ format is not
  implemented. This is simple to do, but interface would need to be though out.

** Implementation details

- [[./core/frames.lisp]] implements frame reading and writing. It is intended to be
  correct and fast, in this order. Read frames call callbacks that are supposed
  to be generic functions for extensibility.
- [[./core/classes.lisp]] defines class hierarchy for vanilla connections and streams,
  and the generic methods for callbacks for these classes.

** Dependencies

(to be reviewed)

The core library used trivial-gray-streams to implement streams over data frames.

Client and server require usocket and cl+ssl to talk over TLS.

Client uses puri to manipulate URLs.

Server also uses cffi directly to [[file:tls/cl+ssl.lisp][check and confirm alpn]]. The sample server uses cl-who to create html; this is an arbitrary choice and your server can use anything else.

Additioanlly, fiasco is used for testing.

** License
Licensed by MIT license.

Some comments are taken over from the RFCs above and copyrighted by RFC
contributors. I read the copyright licenses for RFC that this is allowed. Big
chunks of RFC text mean usually something is not implemented and are slowly
converted to code.

** Related software
There is an Akamai code on https://github.com/akamai/cl-http2-protocol that
supported bigger parts of the drafted HTTP/2 protocol in 2014; apparently hard
to run now, used NPN instead of ALPN. It might be reasonable to pilfer some
pieces and ideas (especially interface level) from that one, but I have not done
so (yet).


flexi-streams

trivial-gray-streams

http2/client

http2/server

http2/test

http2/tls

http2/all

cl+ssl

puri

cl-who

bordeaux-threads

fiasco

Glue to wrap HTTP/2 client or server with TLS

http2

Upstream URL

Author

License

http2

Version:

License:

Authors:

Dependencies:

http2/all

Dependencies:

http2/client

Version:

License:

Authors:

Dependencies:

http2/server

Version:

License:

Authors:

Dependencies:

http2/test

License:

Dependencies:

http2/tls

Version:

License:

Authors:

Dependencies:

Dependencies (7)

Dependents (0)