Buffer definition; audio buffer abstraction for sound synthesis systems

bdef

#+TITLE: bdef

"Buffer definition"; abstraction of audio buffers for Lisp sound synthesis systems.

Basically, this simplifies buffer management in [[https://github.com/byulparan/cl-collider][cl-collider]], making them easier to use.

*Note:* Previously Bdef for SuperCollider was hosted at this URL. That repository has since moved [[https://github.com/defaultxr/supercollider-bdef][here]].

* Intro

Bdef is available from Quicklisp, so you can simply Quickload it:

#+BEGIN_SRC lisp
(ql:quickload :bdef)
#+END_SRC

It also has multiple sub-systems for integration with various other libraries. So if you use cl-patterns, cl-collider, or Incudine you may want to load ~bdef/cl-patterns~ and/or the other sub-systems:

#+BEGIN_SRC lisp
;; integration with cl-patterns and cl-collider:
(ql:quickload '(bdef/cl-patterns bdef/cl-collider))
#+END_SRC

To load a buffer from a file, provide the path to the ~bdef~ function:

#+BEGIN_SRC lisp
(bdef "~/path/to/your/buffer.mp3")

;; you can also give it an alias:
(bdef :foo "/path/to/buffer.wav")

;; the buffer can then be referred to with it:
(bdef :foo)
#+END_SRC

~bdef~ returns a bdef object which can be provided in place of a buffer for any functions that expect it, as long as you've quickloaded the proper bdef sub-system for the library those functions are from. For example:

#+BEGIN_SRC lisp
;; cl-collider:
(synth :playbuf :buffer (bdef :foo)) ; will work if you've loaded the bdef/cl-collider system.

;; cl-patterns:
(play (event :instrument :playbuf :buffer :foo)) ; will work if you've loaded the bdef/cl-patterns system.
;; Notice that you can just specify the bdef name as a symbol without even having to use the bdef function! The same is true in patterns too.
#+END_SRC

Metadata can be associated with bdefs for various uses:

#+BEGIN_SRC lisp
(setf (bdef-metadata :foo :bpm) 120) ; set the "bpm" metadatum to 120
#+END_SRC

Some metadata is automatically generated; for example, the tempo may be automatically detected from the filename or id3 tags.

bdef includes functionality for defining buffer regions called ~splits~. This is useful, for instance, to divide a drum loop up by each hit, or to divide up a source track into sections based on onsets or beats.

#+BEGIN_SRC lisp
;; define three consecutive regions, one second long each:
(make-splits (list (list 0 1) (list 1 2) (list 2 3)) :unit :seconds)

;; auto-generate splits from sound onsets in a file using the aubio library:
(splits-from-aubio-onsets "/path/to/file.wav")
#+END_SRC

[[https://aubio.org][Aubio]] is an external library of audio analysis functions. If installed, splits can be automatically generated from its analyses.

bdef can also generate splits from other formats as well:
- OP-1 drumsets (~splits-from-op-1-drumset~, but automatically parsed from any valid ~aif~ or ~aiff~ file)
- Audacity labels (~splits-from-audacity-labels~)
- Renoise regions (planned for future implementation)

See the following section for detail on more features of the bdef library.

* Features

** Can be re-evaluated without loading a new buffer:

Compare:

#+BEGIN_SRC lisp
  (defparameter *buf* (cl-collider:buffer-read "/buffer.wav"))

  (defparameter *buf* (cl-collider:buffer-read "/buffer.wav")) ; the same variable, and same file!

  (length (remove nil (slot-value *s* 'cl-collider::buffers))) ; => 2 -- duplicate buffers!
#+END_SRC

versus:

#+BEGIN_SRC lisp
  (bdef :buf "/buffer.wav") ; here we give it the name :buf

  (bdef :foo "/buffer.wav") ; same file, different "name"...

  (length (remove nil (slot-value *s* 'cl-collider::buffers))) ; => 1 -- no duplicate buffers :D
#+END_SRC

...To force a file to be reloaded, simply call ~bdef-free~ on it, then call ~bdef~ again.

** Automatically converts files unsupported by the backend if you have ffmpeg installed:

#+BEGIN_SRC lisp
(bdef :bar "/blah.mp3") ; works!
#+END_SRC

It does this by storing them in a temporary directory (~/tmp/bdef/~ by default on linux and mac).

** No additional name needed if loading from a file:

#+BEGIN_SRC lisp
(bdef "/my-file.ogg")
#+END_SRC

** Supports pathname abbreviations:

#+BEGIN_SRC lisp
(bdef "~/cool-sound.wav") ; will find a cool sound in your home directory
#+END_SRC

** Loads mono files as stereo by default.

For consistency. To load as mono, supply 1 for ~bdef~'s ~num-channels~ keyword argument.

** Supports loading in wavetable format:

#+BEGIN_SRC lisp
(bdef "~/wilhelm.wav" :wavetable t) ; load the Wilhelm scream as a wavetable
#+END_SRC

** Supports loading envelopes as buffers:

Either as wavetables, or standard.

** Integration with [[https://github.com/byulparan/cl-collider][cl-collider]]:

#+BEGIN_SRC lisp
(cl-collider:bufnum (bdef :sound)) ; returns the buffer number.

(cl-collider:synth :playbuf :bufnum (bdef :sound)) ; plays the buffer.
#+END_SRC

Load the ~bdef/cl-collider~ system to enable this.

** Integration with [[https://github.com/defaultxr/cl-patterns][cl-patterns]]:

#+BEGIN_SRC lisp
(cl-patterns:play (bdef :sound)) ; plays the buffer using the *cl-collider-buffer-preview-synth* set in cl-patterns.

(cl-patterns:play (cl-patterns:event :instrument :playbuf :bufnum (bdef :sound))) ; automatically converts bdef to the buffer number.
#+END_SRC

Load the ~bdef/cl-patterns~ system to enable this.

** Supports multiple sound server backends:

[[https://supercollider.github.io/][SuperCollider]]/[[https://github.com/byulparan/cl-collider][cl-collider]] is the primary backend tested against, however [[https://incudine.sourceforge.net/][Incudine]] is also supported for most functionality.

Enable the cl-collider backend, for example, like so:

#+BEGIN_SRC lisp
  (ql:quickload :bdef/cl-collider)
#+END_SRC

** Allows metadata about the buffer to be stored:

#+BEGIN_SRC lisp
(setf (bdef-metadata (bdef :snd) :bpm) 99) ; set :snd's tempo to 99 BPM.

(bdef-metadata (bdef :snd) :bpm) ; get the stored bpm value.
#+END_SRC

** Automatically set various metadata when a bdef is created:

#+BEGIN_SRC lisp
;; load a file with its bpm in its filename:
(bdef :my-file "~/my-file-128bpm.wav")

;; the bpm is automatically stored as metadata:
(bdef-metadata :my-file :bpm) ; => 128
#+END_SRC

You can also add your own auto-metadata keys with the ~define-bdef-auto-metadata~ macro or ~set-bdef-auto-metadata~ function, or remove them with the ~remove-bdef-auto-metadata~ function.

Additional metadata is loaded asynchronously in background threads using futures from the [[https://common-lisp.net/project/eager-future/][eager-future2]] library. If a requested metadatum is still being generated, ~bdef-metadata~ will block until the result is available.

** Automatically generate metadata from functions:

#+BEGIN_SRC lisp
(setf (bdef-metadata :foo :bpm) 142) ; sets the "tempo" metadata key instead to its beats per minute value

;; tempo is stored as beats per second:
(bdef-metadata :foo :tempo) ; => 71/30 (142 beats per minute in beats per second)

;; beats per minute is still available, dynamically calculated from the tempo key:
(bdef-metadata :foo :bpm) ; => 142
#+END_SRC

You can define your own "dynamic metadata" with ~define-bdef-dynamic-metadata~.

** "Splits" functionality to define split points or regions in buffers:

#+BEGIN_SRC lisp
  (make-splits (list 0 0.25 0.5 0.75) :bdef (bdef :foo)) ; splits at the start, 25%, 50%, and 75% into the file.

  (splits-from-audacity-labels "/path/to/label.txt") ; make a splits object from an Audacity labels file.

  (setf (bdef-splits :my-bdef) *) ; set the :my-bdef bdef's :splits metadatum to the splits object generated from the above.

  (splits-point :my-bdef 3 :start :second) ; get the start of :my-bdef's fourth split in seconds.
#+END_SRC

** Splits integration with cl-patterns:

#+BEGIN_SRC lisp
  (pbind :instrument :playbuf
         :bufnum (bdef :my-bdef)
         :split (pwhite 0 (1- (splits-length :my-bdef))) ; pick a random split
         :embed (psplits) ; the psplits pattern yields events with :start, :end, and :dur keys to play the split specified by :split from the :splits metadatum of the bdef specified as :bufnum.
         :dur 1)
#+END_SRC

** Integration with the [[https://aubio.org/][Aubio]] audio analysis library if installed:

#+BEGIN_SRC lisp
(bdef::splits-from-aubio-onsets "/path/to/audio/file.wav")

(bdef :pee "/path/to/pee.wav") ; since no BPM is listed in the filename, aubio is used to detect it (if installed)...

(bdef-metadata :pee :tempo) ; ...and it is stored in the bdef's :tempo metadatum! nice!
#+END_SRC

** Ability to import splits from OP-1 drumset file metadata:

#+begin_src lisp
(bdef::splits-from-op-1-drumset "/path/to/op-1-drumset.aif") ; generates a splits by parsing the metadata in the file.
#+end_src

Note that any ~aif~ or ~aiff~ file will automatically be checked for OP-1 metadata, which will be parsed and stored in the ~splits~ bdef metadata key if it is found.

* Backends

Currently, bdef supports SuperCollider via cl-collider as a backend. There is also basic (likely buggy) Incudine support - this will be improved later.

To write your own backend, you will need to implement the following methods on your backend's buffer class:

- ~bdef-backend-supported-file-types~
- ~bdef-backend-load~
- ~bdef-backend-free~
- ~bdef-length~
- ~bdef-sample-rate~
- ~bdef-channels~
- ~bdef-id~ (optional if your backend doesn't use buffer IDs)
- ~bdef-file~ (optional if your backend doesn't keep track of what file a buffer was loaded from)
- ~bdef-frames~

All other functionality is derived from those functions.

For the user's convenience, you might also want to define methods on the ~bdef~ class for the backend's relevant functions; see the bottom of [[file:cl-collider.lisp][cl-collider.lisp]] for an example.

* Future

- Fix the various minor/not-so-minor issues marked with "FIX" in the code.
- We have ~bdef-frames~ to get buffer data; we should have support for setting buffer data as well.
- Support for configurable pathname shortcuts. (i.e. set ~foo~ as a shortcut to ~/a/long/path/name/~, then provide ~"foo/bar.wav"~ instead of ~"/a/long/path/name/bar.wav"~.)
- "Dynamic" splits; i.e. define a set of splits as "this region in four equal-length pieces" rather than all splits being immediately "baked" as specific points.
- Allow importing as ~splits~ from ~.srt~ (subtitle) files, ~.tsv~ (tab-separated values; this seems to be what Audacity uses, and Whisper has an option to export in this format), and ~.vtt~ (WebVTT; similar to ~.srt~)?
- Auto-generate metadata with [[https://librosa.org/][librosa]] ([[https://github.com/librosa/librosa/][github]]) similar to how we do for Aubio.
  https://librosa.org/doc/latest/index.html
  https://librosa.org/doc/latest/tutorial.html
- Rename the ~:cl-collider~ backend to ~:supercollider~ for consistency with cl-patterns.
- Allow allocating an empty buffer by specifying the number of frames as the value.
- Rename ~bdef-frames~ to ~bdef-data~ ? This is what Incudine calls it (~buffer-data~) and it's less ambiguous whether "frames" refers to the contents of the frames, or to the number of frames.


alexandria

eager-future2

jsown

mutility

parse-float

cl-collider

Buffer definition; audio buffer abstraction for sound synthesis systems (with cl-collider functionality)

bdef/cl-collider

cl-patterns

Buffer definition; audio buffer abstraction for sound synthesis systems (with cl-patterns functionality)

bdef/cl-patterns

incudine

Buffer definition; audio buffer abstraction for sound synthesis systems (with Incudine functionality)

bdef

Upstream URL

Author

License

bdef

Version:

License:

Authors:

Dependencies:

bdef/cl-collider

Version:

License:

Authors:

Dependencies:

bdef/cl-patterns

Version:

License:

Authors:

Dependencies:

bdef/incudine

Version:

License:

Authors:

Dependencies:

bdef/tests

License:

Authors:

Dependencies:

Dependencies (8)

Dependents (0)