cl-opencl-utils
2023-02-15
OpenCL utility library built on cl-opencl
cl-opencl-utils is a library of utilities for working with OpenCL,
especially including a Lispified version of OpenCL C. It is built on
top of the cl-opencl library
(https://www.github.com/ghollisjr/cl-opencl).
The examples/ directory has examples showing how to use some of the
included utilities.
Features implemented so far:
* Lispified OpenCL C language
* Complex numbers (full suite of functions in progress, including libcerf)
* Map (make-opencl-mapper), multiple-input single-output mapping
* Reduce (make-opencl-reducer), only associative 2-argument functions at the moment
* Device-side RNG (pcg32 for PCG-XSH-RR 32 bit)
* Function sampling (make-opencl-function-sampler)
* Finite-domain integration (make-opencl-integrator, make-opencl-complex-integrator)
* Splines (make-opencl-spline-form)
* Runge-Kutta 4th order algorithm (make-opencl-rk4)
cl-opencl-utils is GPL3 with the exception of a few snippets of code
that may or may not be able to be promoted to GPL from e.g. the MIT
license. If they can't be promoted then they're still MIT license as
noted above each snippet.
The Lispified OpenCL C language follows a few rules:
1. If a symbol has not been given special meaning, it will be
translated into an all-lowercase string, non-alphanumeric symbols
preserved.
2. If the first element of a list does not contain a symbol with a
special meaning, it is translated into a C-style function call.
E.g., (f x y) ==> f(x,y).
3. Specialized meanings for symbols are defined with defclc, which
causes the #'clc function to generate a different C source code
string result. E.g., (clc `(zerop x)) ==> ((x) == 0) since zerop
has a special meaning already defined.
4. Functions in C that have a Lisp equivalent have their Lisp aliases
defined. E.g., (expt x y) ==> pow(x,y), but (pow x y) ==> pow(x,y)
as well.
5. Variables are declared via the var and vararray operators:
(var x :int 0) ==> int x = 0.
(var f (const :double) 5d0) ==> const double f = 5.0;
(vararray x :double (2 3)) ==> double x[2][3].
(vararray y :double (5) 1 2 3 4 5) ==> double x[5] = {1,2,3,4,5}
I tend to use the CFFI-style keyword symbols for types as it
improves readability, but it's not technically necessary.
6. Blocks are created with progn:
(progn
(var i :int 0)
(var j :int i))
==>
{int i = 0; int j = i;}
7. for, while, and do-while provide for-loops, while-loops and
do-while loops:
(for (var i :int 0) (< i 10) (incf i)
(setf sum (+ sum i)))
==>
for(int i = 0; i < 10; ++i) {
sum = sum + i;
}
(var x :double 1d0)
(var i :int 0)
(while (< i 5)
(incf i)
(setf x (* x 2d0)))
==>
double x = 1.0;
int i = 0;
while(i < 5) {
++i;
x = x*2.0;
}
(do-while (< i 5)
(incf i)
(setf x (* x 2d0)))
==>
do {
++i;
x = x*2.0;
} while(i < 5);
8. Non-lower case terms can be supplied with Lisp strings, whereas
symbols are converted to lowercase.
(function "Hello" :int () (return 1)) ==> int Hello () {return 1;}
Whereas
(function Hello :int () (return 1)) ==> int hello () {return 1;}
This also works with the defclcfun and defclckernel macros:
(defclcfun "AddSomeNumbers" :int
((var x :int)
(var y:int))
(return (+ x y)))
("AddSomeNumbers" 1 2) ==> 3
Whereas
(AddSomeNumbers 1 2) ==> compilation error
9. Typecasting works using either the typecast or coerce operators:
(typecast x :int) ==> ((int) x)
(coerce x :int) ==> ((int) x)
10. Pointers have a type operator, pointer. The address and value
operators return the address or value of their argument:
(var x :int 0)
;; create pointer variable to address of x
(var px (pointer :int)
(address x))
;; get value of x through the pointer px
(value px)
11. Arrays are accessed with aref:
(aref xs 5) ==> x[5]
(aref ys 1 2 3) ==> y[1][2][3]
12. Structs are defined with defstruct:
(defstruct mystruct
(var x :double)
(var y :int))
==>
struct mystruct {
double x;
int y;
};
Members are accessed by member and pmember, member for struct
objects and pmember for pointers to struct objects:
(member s x) ==> s.x
(pmember p x) ==> p->x
Structs can also be defined in a similar way to functions via
defclcstruct, which both defines a CFFI type accessible to Lisp
and an OpenCL C struct of the same size and structure available on
the OpenCL device. The above example could be defined in Lisp via
(defclcstruct mystruct
(:x :double)
(:y :int))
and later referred to as a type in Lispified OpenCL C code, e.g.
(var s (:struct mystruct))
(setf (member s :x) 1d0) ; setf works with members
with the definition for "mystruct" automatically included in the
OpenCL C source string produced by the various compiler functions
listed below. Note that the (:struct structname) form must be
used to refer to structs in Lispified OpenCL C, while support for
bare structs in CFFI is deprecated but still possible. Good
practice is to use (:struct structname) for both. See
examples/struct.lisp.
Also note that as stated below, #'clc de-packages symbols (except
functions, kernels, and structs defined with defclcfun,
defclckernel, and defclcstruct), so members can be defined and
referred to using any package. To match CFFI style, keyword
symbols are recommended for defclcstruct, but in the above
examples you can change the package of any of the slot names and
the same OpenCL C code will result.
To easily support CFFI-Lisp translation, you can use the :class
keyword during struct definition and define methods for
translate-from-foreign-memory etc.:
(defclcstruct (mystruct :class mystruct)
(:x :double)
(:y :int))
(defmethod translate-from-foreign (pointer (type mystruct))
(list :x (foreign-slot-value pointer '(:struct mystruct) :x)
:y (foreign-slot-value pointer '(:struct mystruct) :y)))
;; example usage:
(convert-from-foreign some-cffi-pointer '(:struct mystruct))
==> (list :x some-x-value :y some-y-value)
See
https://www.common-lisp.net/project/cffi/manual/html_node/Foreign-Type-Translators.html
for more information on defining CFFI-Lisp translation methods.
13. Macros can be defined with defclcmacro:
(defclcmacro square (x)
`(* ,x ,x))
(square 2) ==> (* 2 2) ==> 2*2
Macros have already been used to define complex+ and complex- in
math.lisp, for example.
14. Global variables can be defined with defclcglobalvar:
(defclcglobalvar
(var G_X :double 0d0))
==> double g_x 0.0; //note case
(defclcglobalvar
(var "G_X" :double 0d0))
==> double G_X 0.0; //note case
15. Complex numbers are supported by CFFI type (:struct cl_complex).
Note that native Lisp complex numbers are automatically converted
to and from CFFI, so this type can be used with the OpenCL
functions and native complex data can be sent and received. See
examples/complex.lisp.
When in doubt, test a form with the #'clc function to see what OpenCL
C code it produces.
Note that the OpenCL C code generated has many technically unnecessary
parentheses and code blocks from the perspective of well-written C
code, but this is to ensure that all possible use cases from the Lisp
perspective lead to reasonable code. You can have overly specific C
code generated from Lisp and be safe, but ambiguous code can lead to
strange problems, so I went with the safe route.
Note that symbols are essentially de-packaged from the perspective of
defclc and clc, as they are interned into the cl-opencl-utils package
before processing. This allows the use of symbols from any package to
denote Lispified OpenCL C code, which is surprisingly inconvenient to
use when symbols are treated as if they belong to a single package and
you don't want to import all of cl-opencl-utils into the package
you're using. E.g.,
(clc `(cl:+ 2 2)) ==> (2+2)
(clc `(some-package:+ 2 2)) ==> (2+2)
However, there are mechanisms for defining OpenCL C functions and
struct types, and these are package-dependent so as to allow different
utility libraries to define functions without worrying about clashes.
See notes on defclcfun, defclckernel, and defclcstruct.
OpenCL source code is still converted into strings and supplied to the
cl-opencl Lisp API functions like cl-create-program-with-source, but
there are a few options for how to generate the OpenCL C source code
from the Lispified code. The following functions and accompanying
macros aid in this:
* program-source-from-forms-fn: Function to create source code from
top-level forms of Lispified OpenCL C code. You provide the
Lispified code as arguments to this function directly, and any
defined kernels or programs with the defclcfun and defclckernel
macros are automatically added to the source code when they're
referred to by the supplied code.
* program-source-from-forms: Macro version of the same where arguments
aren't evaluated.
* program-source-from-kernels-fn: Creates source code for a program
from just the kernel symbols supplied. The definitions and
necessary previously defined functions are found and included in the
OpenCL C program generated.
* program-source-from-kernels: Macro version where the arguments
aren't evaluated.
For example, a basic hello-world kernel might be produced via:
;; Kernel definition
(defclckernel hello
((var n (global (pointer :uint)))
(var buf (global (pointer :uint))))
(var gid :int (get-global-id 0))
(when (< gid (value n))
(setf (aref buf gid)
gid)))
;; Make kernel
(let* ((platform (first (cl-get-platform-ids)))
(device (first (cl-get-device-ids platform
+CL-DEVICE-TYPE-ALL+)))
(context (cl-create-context platform
(list device)))
(source
(program-source-from-kernels hello))
(program
(cl-create-program-with-source context source)))
(cl-build-program-with-log program
(list device))
(cl-create-kernel program "hello"))
I recommend reading the examples at least once to get an idea of how
to use the utilities in a complementary way.