This is a persisted, in memory (lazy loading) document store for Common Lisp.
I have tested this version with live systems but am still rolling it out to all my live systems so there might be some bug churn over the next couple of weeks as this version is put through its paces.
cl-naive-store is a database (document store to more precise) which is persisted, in memory (lazy loading), indexed and written completely in Common Lisp.
The naive bit comes from the fact that data is persisted as plists in files to make them human and machine readable, also there is no rocket science code.
The store was designed to be customisable, just about anything can be customised, have a look at the implementation api to get an idea of what is possible.
MIT the doc is here.
What was supposed to take weeks ended up taking months, but we are finally there. The 2021.5.18 version adds sharding to cl-naive-store and as a consequence of that the use of parallel processing internally and thread safety over all (fingers crossed!).
There is more refinement that we would like to do in the future for the parallel processing but what we have now is fast enough and simple enough for a good start.
All the documentation from the wiki has been placed into a docs/ folder as org-mode files.
The latest version was financially supported by Managing Transformation Solutions (Pty) Ltd, thank you for your continued support of open source software!
I must also give a big shout out to Pascal J. Bourguignon for code reviews, threading code, test code and a whole lot more. His guidance was invaluable, even if I did not always listen to him. /Pascal does not endorse this software in any way, shape or form, he assisted me when and where I asked for specific help./
Data can be written to file per document update or as a batch update.
Data is loaded into memory for querying and lookups, that makes them fast.
Data is only loaded when needed and not before. If you use the store correctly it means that you will only have the data that users requested up to that point in memory.
Documents and key values can be indexed, and a user can specify their own additional indexes as well. Queries and lookups can both be done using indexes which speeds up the retrieval of data considerably.
Sharding is the breaking down of files into smaller files, in the case of naive-store that means that instead of one file per collection there could be many.
Sharding is done based on the actual data in collections. The user specifies which elements of a document it wants to use for sharding on a collection. If none is specified no sharding is done.
cl-naive-store can do a lot but you as the user decides how much of the store's functionality you want to use for your own project.
Functionality was broken down into these packages:
The following .asd files can be used to load different functionality:
- cl-naive-store.naive-core.asd loads the most basic functionality forcl-naive-store. Use this if you don't any of the other extensions.
- cl-naive-store.naive-merkle.asd loads naive-documents and theexperimental merkle functionality.
- cl-naive-store.naive-indexed.asd loads naive-core and indexfunctionality.
- cl-naive-store.document-types.asd loads naive-core and document-typefunctionality.
- cl-naive-store.document-defs.asd loads naive-core, document-typesand type definition functionality.
- cl-naive-store.documents.asd loads naive-core, naive-indexed,documents-types, document-type-defs and document functionality.
- cl-naive-store.asd loads the whole shebang.
- cl-naive-store.test.asd loads tests
Documentation can be found in the docs folder in the repository.
Examples are in the examples folder in the git repository. If those are to simplistic for you have a look at the code in the tests.
1.8Supported CL Implementations
All Tests pass on SBCL an CCL
Have a look at the issues in gitlab, future development is tagged accordingly.
Go to the tests folder
Run make with any of the following
You should see the following at the end.
SUCCESS COUNT: 46 FAILURE COUNT: 0 TOTAL TESTS: 46 Completed Test CL-NAIVE-STORE-TESTS:TEST-ALL