cl-jsonl
2023-06-18
Lazy read JSONL files with each line a separate definition.
Efficient JSON File Iteration
Lazy read JSONL files with each line a separate definition. JSONL files are files with newline delimited JSON blobs and typically very large. This package provides a generator that allows a line by line parse reducing memory complexity for large data demand projects such as those for training deep learning models.
Usage
;; load libraries and import dependency symbols to the current system (ql:quickload :cl-jsonl :silent t) (ql:quickload :gtwiwtg :silent t) (use-package :cl-jsonl) (use-package :gtwiwtg) ;; parse the first line of JSON in file `test.json' (with-json-reader (g #p"test.json") (format t "~S~%" (take 1 g)))
Obtaining
The easiest way to install is using quicklisp:
(ql:quickload :cl-jsonl)
Changelog
An extensive changelog is available here.
License
Copyright (c) 2023 Paul Landes