Michael Weber: Random Bits and Pieces

...Please don't assume Lisp is only useful for Animation and Graphics, AI, Bioinformatics, B2B and E-Commerce, Data Mining, EDA/Semiconductor applications, Expert Systems, Finance, Intelligent Agents, Knowledge Management, Mechanical CAD, Modeling and Simulation, Natural Language, Optimization, Research, Risk Analysis, Scheduling, Telecom, and Web Authoring just because these are the only things they happened to list.

Kent M. Pitman

Lisp Logo (by Conrad Barsky)

Did you ever want to recreate the source form of a package, to see what state it is in currently? With defpackage-form, you can!

Then again, I seem to fiddle too much with packages lately. In particular, currently I am experimenting with a new work flow of Lisp package management.

Package Forms during Development

When writing new Lisp code, I start with just the essentials of a new package in my current Emacs buffer:

(defpackage #:foo
  (:use #:cl))

(This is basically what I get with C-x C-r P RET RET RET.)

During development, I find it distracting to remind myself of keeping the package form up-to-date. When I decide to import new symbols or shadow others I manipulate the package object directly, instead of updating the above DEFPACKAGE form and reevaluating it: (import 'bar:baz), (shadow 'qux), (use-package 'cl-fred), etc.. All this can be automated with a few editor key bindings, and the effects are visible in my working environment immediately.

At this point, I seldom bother with export lists, because I might change names around later on, and also I am working mostly in the package.

At the end of the hacking session, I can just evaluate (defpackage-form 'foo) to get the current state of package FOO conveniently as a DEFPACKAGE form, which I can then use to replace the initial stub. Eventually, if the code grows big enough to split it up, I move the package form to a separate file packages.lisp.

At least in the first iteration, the export list is likely still incomplete, so I can choose to include all symbols whose home package is the package I am working on, nicely sorted by name. Then I remove those symbols which are not meant for exporting. Like that, it is easy to bulk export many symbols at once. As a common convention, symbols starting with % are meant to be internal. They can be omitted from the export list automatically.

FORMAT Controls, the Ultimate Line Noise

Package forms print neatly (and by that I mean closer to how I would layout them manually) with the following additions:

(defun pprint-defpackage (stream defpackage-form)
  (format stream "~:<~W~^ ~3I~:_~W~^~1I~@{~:@_~:<~W~^ ~:I~@_~@{~W~^ ~_~}~:>~}~:>"

(set-pprint-dispatch '(cons (member defpackage))

Current versions of SBCL (newer than come with this style included already. I also toyed with a two-column layout, but it is harder to edit.

Incremental Updates

Incremental updates of package forms are a little trickier, though. The generated form could, for example, be compared to the original with diff-sexp. I have not automated that stage much yet. If there are only few additions, it is probably easier to just add them manually.

Otherwise, I could imagine an Emacs before-save-hook which checks whether the export list of a package matches the DEFPACKAGE form in packages.lisp (or wherever it is stored—this could be figured out with source locations), and some more automation and integration with Emacs.

Very unfortunately, some loss of information occurs on the round trip from package form to object and back:

  • Reader conditionals inside DEFPACKAGE forms cannot be recreated easily. Neither can comments, for what it's worth.
  • There appears to be no way to figure out which package a symbol was actually imported from. For example, if we export CL:NIL from package FOO and then import FOO:NIL into some other package BAR, it will appear the same as if CL:NIL was imported directly into BAR.
  • The default :USE list (i.e., when none is specified) is implementation dependent, which naturally can cause some implementation dependent packages to show up in this list.