1cl-naive-store

cl-naive-store is a database (document store to more precise) which is persistent, in-memory (lazily-loaded), indexed, and written completely in Common Lisp.

The "naive" comes from the fact that data is persisted as plists in files to make them human and machine readable, also there is no rocket science code.

The store was designed to be customisable, just about anything can be customised, have a look at the implementation API to get an idea of what is possible.

1.1Status

The package is considered stable.

Over the next couple of months the package will be used in commercial software and should get a good shake down during that process.

1.2License

cl-naive-store is released under the MIT license.

1.3News

cl-naive-store.utils was added to help with loading database from definitions.

1.4Acknowledgements

I must also give a big shout out to Pascal J. Bourguignon for code reviews, threading code, test code and a whole lot more. His guidance was invaluable, even if I did not always listen to him. /Pascal does not endorse this software in any way, shape or form, he assisted me when and where I asked for specific help./

1.5Features

1.5.1Persisted

Data can be written to file for each document update, or as a batch update.

1.5.2In memory

Data is loaded into memory for querying and lookups, which makes these operations fast.

1.5.3Lazy Loading

Data is only loaded when needed. If you use the store correctly it means that you will only have the data that users requested up to that point in memory.

1.5.4Indexed

Documents and key-values can be indexed, and a user can specify their own additional indexes as well. Queries and lookups can both be done using indexes which speeds up the retrieval of data considerably.

1.5.5Sharding

Sharding is the breaking down of files into smaller files, in the case of naive-store that means that instead of one file per collection there could be many.

Sharding is done based on the actual data in collections. The user specifies which elements of a document it wants to use for sharding on a collection. If none is specified no sharding is done.

1.5.6Layered Design

cl-naive-store can do a lot but you as the user decides how much of the store's functionality you want to use for your own project.

Functionality was broken down into these packages:

• cl-naive-store.naive-core
• cl-naive-store.document-types
• cl-naive-store.document-type-defs
• cl-naive-store.naive-documents
• cl-naive-store.naive-indexed
• cl-naive-store.naive-merkle
• cl-naive-store.test

The following .asd files can be used to load different functionality:

• cl-naive-store.naive-core.asd loads the most basic functionality forcl-naive-store. Use this if you don't any of the other extensions.
• cl-naive-store.naive-merkle.asd loads naive-documents and theexperimental merkle functionality.
• cl-naive-store.naive-indexed.asd loads naive-core and indexfunctionality.
• cl-naive-store.document-types.asd loads naive-core and document-typefunctionality.
• cl-naive-store.document-defs.asd loads naive-core, document-typesand type definition functionality.
• cl-naive-store.documents.asd loads naive-core, naive-indexed,documents-types, document-type-defs and document functionality.
• cl-naive-store.asd loads the whole shebang.
• cl-naive-store.test.asd loads tests

1.6Documentation

The documentation can be found in the docs folder in the repository.

1.7Examples

Examples are in the examples folder in the git repository. If those are to simplistic for you have a look at the code in the tests.

1.8Dependencies

• cl-fad
• iron-clad
• cl-murmurhash
• split-sequence
• uuid
• local-time
• cl-getx
• bordeaux-threads
• lparallel
• cl-cpus

1.9Supported CL Implementations

All tests pass on SBCL an CCL

1.10Development Roadmap

Have a look at the Gitlab issue tracker, future development is tagged accordingly.

1.11Tests

Go to the tests folder

cd cl-naive-store/

Run make with any of the following

• test
• test-load-systems
• test-run-tests
• run-tests-ccl
• run-tests-sbcl

For example:

make test-run-tests

You should see the following at the end.

SUCCESS COUNT:       46
FAILURE COUNT:        0
TOTAL TESTS:         46
Completed Test CL-NAIVE-STORE-TESTS:TEST-ALL