cl-cxx-jit

#+OPTIONS: toc:3          (only include two levels in TOC)
* CL-CXX-JIT - Common Lisp C++ JIT for exposing C++ functions

This library provides an interface to C++ from lisp. It compiles C++ code, then loads it into lisp. It was inspired by [[https://github.com/onqtam/rcrl][RCRL]] and the older project [[https://github.com/Islam0mar/cl-cxx][CL-CXX]].

** API
=(from '("list of string headers") 'import "normal string is inserted as it is" '("function/function-pointer/mem-fun-pointer/lambda" . "name-used-in-lisp"))=

** Examples
*** SDL2 Example
[[sdl2.gif]]
#+begin_src lisp
(ql:quickload :cxx-jit)
(setf cxx-jit:*cxx-compiler-link-libs* "-lGL -lSDL2 -lSDL2main")

(cxx-jit:from '(
                "<SDL2/SDL.h>")
              'import
              '("[](){return SDL_Init(SDL_Init(SDL_INIT_VIDEO));}" . "init")
              '("SDL_CreateWindow" . "create-window")
              '("SDL_CreateRenderer" . "create-renderer")
              '("SDL_SetRenderDrawColor" . "set-color")
              '("SDL_DestroyWindow" . "destroy-window")
              '("SDL_RenderClear" . "clear-renderer")
              '("SDL_RenderPresent" . "renderer-render")
              '("SDL_Quit" . "sdl-quit"))

(init)
(setf wind (create-window "create-window" 0 0 600 700 0))
(setf rend (create-renderer wind -1 0))
(loop for x to (* 255 3)
      for r = (if (> x 255) 255 x)
      for g = (if (> x 255) (if (> x (* 2 255)) 255 (rem x 256)) 0)
      for b = (if (> x (* 2 255)) (rem x 256) 0)
      do
         (print x)
         (set-color rend r g b 255)
         (clear-renderer rend)
         (renderer-render rend)
         (sleep 0.01))

(destroy-window wind)
(sdl-quit)
#+end_src

*** Basic Example
Start with =(ql:quickload :cxx-jit) (in-package cxx-jit)=
#+begin_src lisp
  (from '("<string>") 'import '("[](std::string x){return \"Hi, \"+x;}" . "hi"))
  (hi "there!")

  (from '("<cmath>") 'import '("static_cast<double(*)(double)>(std::sin)" . "cpp-sin"))
  (cpp-sin 0d0)
  (cpp-sin pi)
  (cpp-sin (/ pi 2))

  (from nil 'import "struct C{ auto hi(){return \"Hello, World\\n\";} auto bye(){return \"Bye\";} };" '("&C::bye" . "bye") '("&C::hi" . "hi") '("[](){static C x; return x;}" . "cc"))
  (cc)
  (hi *)
  (bye **)
    ;;; structure  definition could be written in a header file then be used as the following:
  (from '("c.hpp") 'import '("&C::bye" . "bye") '("&C::hi" . "hi") '("[](){static C x; return x;}" . "cc"))
#+end_src

*** Eigen Library Example
Start with =(ql:quickload :cxx-jit) (in-package cxx-jit)=
#+begin_src lisp
  (from '("<Eigen/Core>" "<Eigen/Core>") 'import '("[](Eigen::CwiseNullaryOp<Eigen::internal::scalar_identity_op<double>,Eigen::Matrix<double, 3, 3>> x){
    std::stringstream s;
    s << x;
    return s.str();}" . "print-matrix"))

  (from '("<Eigen/Core>" "<Eigen/Core>") 'import '("static_cast<const Eigen::CwiseNullaryOp<Eigen::internal::scalar_identity_op<double>,Eigen::Matrix<double, 3, 3>> (*)()> (&Eigen::Matrix3d::Identity)" . "identity-matrix"))

  (print-matrix (identity-matrix))
#+end_src

** Prerequisites
 - common lisp supporting [[https://www.common-lisp.net/project/cffi/][CFFI]]
 - working C++17 compiler
 - the following variables should be set according to OS and compiler used
|---------------------------------------------+------------------------------------------------------------------------------------------------|
| variable                                    | default value                                                                                  |
|---------------------------------------------+------------------------------------------------------------------------------------------------|
| =*cxx-compiler-executable-path*=            | "~/usr/bin/g++~"                                                                               |
| =*cxx-compiler-flags*=                      | "~-std=c++17 -Wall -Wextra -I/usr/include/eigen3~"                                             |
| =*cxx-compiler-working-directory*=          | "~/tmp/~" #\/ '/' should be the last character                                                 |
| =+cxx-compiler-lib-name+=                   | (intern "~plugin~")                                                                            |
| =+cxx-compiler-wrap-cxx-path+=              | shouldn't be changed "~path to wrap-cxx.cpp~"                                                  |
| =*cxx-compiler-internal-flags*=             | "~-shared -fPIC -Wl,--no-undefined -Wl,--no-allow-shlib-undefined~"                            |
|                                             | for g++ and for clang++ "~-shared -fPIC -Wl,-undefined,error -Wl,-flat_namespace~"             |
| =*cxx-compiler-link-libs*=                  | "~-lm~" these flags are added after "~-o output~" to link correctly                            |
| =*cxx-type-name-to-cffi-type-symbol-alist*= | alist to map c++ type name to cffi type example:"~(push (cons "long unsigned int" :ulong) *)~" |
|---------------------------------------------+------------------------------------------------------------------------------------------------|

** Installation
Clone into home/common-lisp directory. Then =(ql:quickload :cxx-jit-test)=

** Supported Types
|------------------+------------------|
| C++ type         | Lisp cffi type   |
|------------------+------------------|
| fundamental      | same             |
| string           | :string          |
| class            | :pointer         |
| std::is_function | :pointer         |
| other            | not implemented! |
|------------------+------------------|

** Under The Hood
 + function/lambda/member_function/function_pointer is wrapped into a dummy lambda class to have a unique template specialization.
     #+begin_src C++ 
       Import([&]() { return __VA_ARGS__; });
      #+end_src
 + =Import= function calls =DecayThenResolve= with function pointer as the template specialization so thunk pointer is omitted and we only return the direct function pointer which will be used from lisp side.
 + =InvocableTypeName= returns a vector contains: [return type, class type for class function member, args]. It resolves C++ types as follows:
   * Fundamental types and pointers are passed directly
   * String is converted to char* with new[] operator, should be cleared with =ClCxxDeleteObject(ptr, true)=
   * Class/std::is_function is converted to void* with new[] operator, should be cleared with =ClCxxDeleteObject(ptr, false)=
   * rest report an issue for other cases
 + Meta data for each function defined is passed through a lisp callback with this data:
    #+begin_src C++
typedef struct {
  // could be void*
  void (*thunk_ptr)();
  bool method_p;
  const char **type;  // memory handled in C++
  std::uint8_t type_size;
} MetaData;
#+end_src

** NOTE
Tested on:
 - SBCL 2.0.1 on debian

** Todo List
*** TODO Add redirect stdout : =freopen("/tmp/tmp.txt", "w", stdout);= @apemangr
*** TODO Use trivial-garbage with =ClCxxDeleteObject=
*** TODO Add non-polling =from=
*** TODO Test functions
*** TODO Benchmark
*** TODO Better class interface

** Copyright

Copyright (c) 2021 Islam Omar (io1131@fayoum.edu.eg)

** License

Licensed under the MIT License.


cffi

trivial-garbage

uiop

#+OPTIONS: toc:3          (only include two levels in TOC)
* CL-CXX-JIT - Common Lisp C++ JIT for exposing C++ functions

This library provides an interface to C++ from lisp. It compiles C++ code, then loads it into lisp. It was inspired by [[https://github.com/onqtam/rcrl][RCRL]] and the older project [[https://github.com/Islam0mar/cl-cxx][CL-CXX]].

** API
=(from '("list of string headers") 'import "normal string is inserted as it is" '("function/function-pointer/mem-fun-pointer/lambda" . "name-used-in-lisp"))=

** Examples
*** SDL2 Example
[[sdl2.gif]]
#+begin_src lisp
(ql:quickload :cxx-jit)
(setf cxx-jit:*cxx-compiler-link-libs* "-lGL -lSDL2 -lSDL2main")

(cxx-jit:from '(
                "<SDL2/SDL.h>")
              'import
              '("[](){return SDL_Init(SDL_Init(SDL_INIT_VIDEO));}" . "init")
              '("SDL_CreateWindow" . "create-window")
              '("SDL_CreateRenderer" . "create-renderer")
              '("SDL_SetRenderDrawColor" . "set-color")
              '("SDL_DestroyWindow" . "destroy-window")
              '("SDL_RenderClear" . "clear-renderer")
              '("SDL_RenderPresent" . "renderer-render")
              '("SDL_Quit" . "sdl-quit"))

(init)
(setf wind (create-window "create-window" 0 0 600 700 0))
(setf rend (create-renderer wind -1 0))
(loop for x to (* 255 3)
      for r = (if (> x 255) 255 x)
      for g = (if (> x 255) (if (> x (* 2 255)) 255 (rem x 256)) 0)
      for b = (if (> x (* 2 255)) (rem x 256) 0)
      do
         (print x)
         (set-color rend r g b 255)
         (clear-renderer rend)
         (renderer-render rend)
         (sleep 0.01))

(destroy-window wind)
(sdl-quit)
#+end_src

*** Basic Example
Start with =(ql:quickload :cxx-jit) (in-package cxx-jit)=
#+begin_src lisp
  (from '("<string>") 'import '("[](std::string x){return \"Hi, \"+x;}" . "hi"))
  (hi "there!")

  (from '("<cmath>") 'import '("static_cast<double(*)(double)>(std::sin)" . "cpp-sin"))
  (cpp-sin 0d0)
  (cpp-sin pi)
  (cpp-sin (/ pi 2))

  (from nil 'import "struct C{ auto hi(){return \"Hello, World\\n\";} auto bye(){return \"Bye\";} };" '("&C::bye" . "bye") '("&C::hi" . "hi") '("[](){static C x; return x;}" . "cc"))
  (cc)
  (hi *)
  (bye **)
    ;;; structure  definition could be written in a header file then be used as the following:
  (from '("c.hpp") 'import '("&C::bye" . "bye") '("&C::hi" . "hi") '("[](){static C x; return x;}" . "cc"))
#+end_src

*** Eigen Library Example
Start with =(ql:quickload :cxx-jit) (in-package cxx-jit)=
#+begin_src lisp
     (from '("<Eigen/Core>" "<Eigen/Core>") 'import '("[](Eigen::CwiseNullaryOp<Eigen::internal::scalar_identity_op<double>,Eigen::Matrix<double, 3, 3>> x){std::stringstream s;
         s << x;
         return s.str();}" . "print-matrix"))

    (from '("<Eigen/Core>" "<Eigen/Core>") 'import '("static_cast<const Eigen::CwiseNullaryOp<Eigen::internal::scalar_identity_op<double>,Eigen::Matrix<double, 3, 3>> (*)()> (&Eigen::Matrix3d::Identity)" . "identity-matrix"))

(print-matrix (identity-matrix))
#+end_src

** Prerequisites
 - common lisp supporting [[https://www.common-lisp.net/project/cffi/][CFFI]]
 - working C++17 compiler
 - the following variables should be set according to OS and compiler used
|------------------------------------+----------------------------------------------------------------------------|
| variable                           | default value                                                              |
|------------------------------------+----------------------------------------------------------------------------|
| =*cxx-compiler-executable-path*=   | "~/usr/bin/g++~"                                                           |
| =*cxx-compiler-flags*=             | "~-std=c++17 -Wall -Wextra -I/usr/include/eigen3~"                         |
| =*cxx-compiler-working-directory*= | "~/tmp/~" #\/ '/' should be the last character                             |
| =+cxx-compiler-lib-name+=          | "~plugin~"                                                                 |
| =+cxx-compiler-wrap-cxx-path+=     | shouldn't be changed "~path to wrap-cxx.cpp~"                              |
| =*cxx-compiler-internal-flags*=    | "~-shared -fPIC -Wl,--no-undefined -Wl,--no-allow-shlib-undefined~"        |
|                                    | for g++ and for clang++ "~-shared -fPIC -undefined error -flat_namespace~" |
| =*cxx-compiler-link-libs*=         | "~-lmath~" these flags are added after "~-o output~" to link correctly     |
|------------------------------------+----------------------------------------------------------------------------|

** Installation
Clone into home/common-lisp directory. Then =(ql:quickload :cxx-jit-test)=

** Supported Types
|------------------+------------------|
| C++ type         | Lisp cffi type   |
|------------------+------------------|
| fundamental      | same             |
| string           | :string          |
| class            | :pointer         |
| std::is_function | :pointer         |
| other            | not implemented! |
|------------------+------------------|

** Under The Hood
 + function/lambda/member_function/function_pointer is wrapped into a dummy lambda class to have a unique template specialization.
     #+begin_src C++ 
       Import([&]() { return __VA_ARGS__; });
      #+end_src
 + =Import= function calls =DecayThenResolve= with function pointer as the template specialization so thunk pointer is omitted and we only return the direct function pointer which will be used from lisp side.
 + =InvocableTypeName= returns a vector contains: [return type, class type for class function member, args]. It resolves C++ types as follows:
   * Fundamental types and pointers are passed directly
   * String is converted to char* with new[] operator, should be cleared with =ClCxxDeleteObject(ptr, true)=
   * Class/std::is_function is converted to void* with new[] operator, should be cleared with =ClCxxDeleteObject(ptr, false)=
   * rest report an issue for other cases
 + Meta data for each function defined is passed through a lisp callback with this data:
    #+begin_src C++
typedef struct {
  // could be void*
  void (*thunk_ptr)();
  bool method_p;
  const char **type;  // memory handled in C++
  std::uint8_t type_size;
} MetaData;
#+end_src

** NOTE
Tested on:
 - SBCL 2.0.1 on debian

** Todo List
*** TODO Use trivial-garbage with =ClCxxDeleteObject=
*** TODO Add non-polling =from=
*** TODO Test functions
*** TODO Benchmark
*** TODO Better class interface

** Copyright

Copyright (c) 2021 Islam Omar (io1131@fayoum.edu.eg)

** License

Licensed under the MIT License.


cl-cxx-jit

Upstream URL

Author

License

cxx-jit

Version:

License:

Authors:

Dependencies:

Dependencies (4)

Dependents (0)