cl-naive-store
2023-06-18
This is a persisted, in memory (lazy loading) document store for Common Lisp.
Upstream URL
Author
License
1cl-naive-store
cl-naive-store
is a database (document store to more precise) which is
persistent, in-memory (lazily-loaded), indexed, and written completely in
Common Lisp.
The "naive" comes from the fact that data is persisted as plists in files to make them human and machine readable, also there is no rocket science code.
The store was designed to be customisable, just about anything can be customised, have a look at the implementation API to get an idea of what is possible.
1.1Status
The package is considered stable.
Over the next couple of months the package will be used in commercial software and should get a good shake down during that process.
1.2License
cl-naive-store
is released under the MIT license.
1.3News
cl-naive-store.utils
was added to help with loading database from
definitions.
1.4Acknowledgements
I must also give a big shout out to Pascal J. Bourguignon for code reviews, threading code, test code and a whole lot more. His guidance was invaluable, even if I did not always listen to him. /Pascal does not endorse this software in any way, shape or form, he assisted me when and where I asked for specific help./
1.5Features
1.5.1Persisted
Data can be written to file for each document update, or as a batch update.
1.5.2In memory
Data is loaded into memory for querying and lookups, which makes these operations fast.
1.5.3Lazy Loading
Data is only loaded when needed. If you use the store correctly it means that you will only have the data that users requested up to that point in memory.
1.5.4Indexed
Documents and key-values can be indexed, and a user can specify their own additional indexes as well. Queries and lookups can both be done using indexes which speeds up the retrieval of data considerably.
1.5.5Sharding
Sharding is the breaking down of files into smaller files, in the case of naive-store that means that instead of one file per collection there could be many.
Sharding is done based on the actual data in collections. The user specifies which elements of a document it wants to use for sharding on a collection. If none is specified no sharding is done.
1.5.6Layered Design
cl-naive-store
can do a lot but you as the user decides how much of
the store's functionality you want to use for your own project.
Functionality was broken down into these packages:
cl-naive-store.naive-core
cl-naive-store.document-types
cl-naive-store.document-type-defs
cl-naive-store.naive-documents
cl-naive-store.naive-indexed
cl-naive-store.naive-merkle
cl-naive-store.test
The following .asd files can be used to load different functionality:
cl-naive-store.naive-core.asd
loads the most basic functionality forcl-naive-store
. Use this if you don't any of the other extensions.cl-naive-store.naive-merkle.asd
loadsnaive-documents
and theexperimental merkle functionality.cl-naive-store.naive-indexed.asd
loadsnaive-core
and indexfunctionality.cl-naive-store.document-types.asd
loadsnaive-core
and document-typefunctionality.cl-naive-store.document-defs.asd
loadsnaive-core
, document-typesand type definition functionality.cl-naive-store.documents.asd
loads naive-core, naive-indexed,documents-types, document-type-defs and document functionality.cl-naive-store.asd
loads the whole shebang.cl-naive-store.test.asd
loads tests
1.6Documentation
The documentation can be found in the docs folder in the repository.
1.7Examples
Examples are in the examples folder in the git repository. If those are to simplistic for you have a look at the code in the tests.
1.8Dependencies
- cl-fad
- iron-clad
- cl-murmurhash
- split-sequence
- uuid
- local-time
- cl-getx
- bordeaux-threads
- lparallel
- cl-cpus
1.9Supported CL Implementations
All tests pass on SBCL an CCL
1.10Development Roadmap
Have a look at the Gitlab issue tracker, future development is tagged accordingly.
1.11Tests
Go to the tests folder
cd cl-naive-store/
Run make with any of the following
- test
- test-load-systems
- test-run-tests
- run-tests-ccl
- run-tests-sbcl
For example:
make test-run-tests
You should see the following at the end.
SUCCESS COUNT: 46 FAILURE COUNT: 0 TOTAL TESTS: 46 Completed Test CL-NAIVE-STORE-TESTS:TEST-ALL