-*- mode: text; coding: utf-8-unix; -*- ###################################################################### ## ## Copyright (C) 2001,2000, 2003 ## Department of Computer Science, University of Tromsø, Norway ## ## Filename: README ## Author: Frode Vatvedt Fjeld <frodef@acm.org> ## Created at: Wed Dec 8 15:35:53 1999 ## Distribution: See the accompanying file COPYING. ## ## $Id: README,v 1.1.1.1 2004/01/13 11:13:13 ffjeld Exp $ ## ###################################################################### Binary-types is a Common Lisp package for reading and writing binary files. Binary-types provides macros that are used to declare the mapping between lisp objects and some binary (i.e. octet-based) representation. Supported kinds of binary types include: * Signed and unsigned integers of any octet-size, big-endian or little-endian. Maps to lisp integers. * Enumerated types based on any integer type. Maps to lisp symbols. * Complex bit-field types based on any integer type. Sub-fields can be numeric, enumerated, or bit-flags. Maps to lisp lists of symbols and integers. * Fixed-length and null-terminated strings. Maps to lisp strings. * Compound records of other binary types. Maps to lisp DEFCLASS classes or, when you prefer, DEFSTRUCT structs. Typically, a complete binary record format/type can be specified in a single (nested) declaration statement. Such compound records may then be read and written with READ-BINARY and WRITE-BINARY. Binary-types is *not* helpful in reading files with variable bit-length code-words, such as most compressed file formats. It will basically only work with file-formats based on 8-bit bytes (octets). Also, at this time no floating-point types are supported out of the box. Binary types may now be declared with the DEFINE-BINARY-CLASS macro, which has the same syntax (and semantics) as DEFCLASS, only there is an additional slot-option (named :BINARY-TYPE) that declares that slot's binary type. Note that the binary aspects of slots are *not* inherited (the semantics of inheriting binary slots is unclear to me). Another slot-option added by binary-types is :MAP-BINARY-WRITE, which names a function (of two arguments) that is applied to the slot's value and the name of the slot's binary-type in order to obtain the value that is actually passed to WRITE-BINARY. Similarly, :MAP-BINARY-READ takes a function that is to be applied to the binary data and type-name when a record of that type is being read. A slightly modified version of :map-binary-read is :MAP-BINARY-READ-DELAYED, which will do essentially the same thing as :map-binary-read, only the mapping will be "on-demand": A slot-unbound method will be created for this purpose. A variation of the :BINARY-TYPE slot-option is :BINARY-LISP-TYPE, which does everything :BINARY-TYPE does, but also passes on a :TYPE slot-option to DEFCLASS (or DEFSTRUCT). The type-spec is inferred from the binary-type declaration. When using this mechanism, you should be careful to always provide a legal value in the slot (as you must always do when declaring slots' types). If you find this confusing, just use :BINARY-TYPE. Performance has not really been a concern for me while designing this package. There's no obvious performance bottlenecks that I know of, but keep in mind that all "binary" reads and writes are reduced to individual 8-bit READ-BYTEs and WRITE-BYTEs. If you do identify particular performance bottlenecks, let me know. The included file "example.lisp" demonstrates how to use this package. To give you a taste of what it looks like, the following declarations are enough to read the header of an ELF executable file with the form (let ((*endian* :big-endian)) (read-binary 'elf-header stream) ;;; ELF basic type declarations (define-unsigned word 4) (define-signed sword 4) (define-unsigned addr 4) (define-unsigned off 4) (define-unsigned half 2) ;;; ELF file header structure (define-binary-class elf-header () ((e-ident :binary-type (define-binary-struct e-ident () (ei-magic nil :binary-type (define-binary-struct ei-magic () (ei-mag0 0 :binary-type u8) (ei-mag1 #\null :binary-type char8) (ei-mag2 #\null :binary-type char8) (ei-mag3 #\null :binary-type char8))) (ei-class nil :binary-type (define-enum ei-class (u8) elf-class-none 0 elf-class-32 1 elf-class-64 2)) (ei-data nil :binary-type (define-enum ei-data (u8) elf-data-none 0 elf-data-2lsb 1 elf-data-2msb 2)) (ei-version 0 :binary-type u8) (padding nil :binary-type 1) (ei-name "" :binary-type (define-null-terminated-string ei-name 8)))) (e-type :binary-type (define-enum e-type (half) et-none 0 et-rel 1 et-exec 2 et-dyn 3 et-core 4 et-loproc #xff00 et-hiproc #xffff)) (e-machine :binary-type (define-enum e-machine (half) em-none 0 em-m32 1 em-sparc 2 em-386 3 em-68k 4 em-88k 5 em-860 7 em-mips 8)) (e-version :binary-type word) (e-entry :binary-type addr) (e-phoff :binary-type off) (e-shoff :binary-type off) (e-flags :binary-type word) (e-ehsize :binary-type half) (e-phentsize :binary-type half) (e-phnum :binary-type half) (e-shentsize :binary-type half) (e-shnum :binary-type half) (e-shstrndx :binary-type half))) For a second example, here's an approach to supporting floats: (define-bitfield ieee754-single-float (u32) (((:enum :byte (1 31)) positive 0 negative 1) ((:numeric exponent 8 23)) ((:numeric significand 23 0)))) The postscript file "type-hierarchy.ps" shows the binary types hierarchy. It is generated using psgraph and closer-mop, which may be loaded via Quicklisp as shown below: (ql:quickload "psgraph") (ql:quickload "closer-mop") (with-open-file (*standard-output* "type-hierarchy.ps" :direction :output :if-exists :supersede) (psgraph:psgraph *standard-output* 'binary-types::binary-type (lambda (p) (mapcar #'class-name (closer-mop:class-direct-subclasses (find-class p)))) (lambda (s) (list (symbol-name s))) t))