Michael Weber: Random Bits and Pieces

Lisp Logo (by Conrad Barsky)

Did you ever want to recreate the source form of a package, to see what state it is in currently? With defpackage-form, you can!

Then again, I seem to fiddle too much with packages lately. In particular, currently I am experimenting with a new work flow of Lisp package management.

Package Forms during Development

When writing new Lisp code, I start with just the essentials of a new package in my current Emacs buffer:


(defpackage #:foo
  (:use #:cl))

(This is basically what I get with C-x C-r P RET RET RET.)

During development, I find it distracting to remind myself of keeping the package form up-to-date. When I decide to import new symbols or shadow others I manipulate the package object directly, instead of updating the above DEFPACKAGE form and reevaluating it: (import 'bar:baz), (shadow 'qux), (use-package 'cl-fred), etc.. All this can be automated with a few editor key bindings, and the effects are visible in my working environment immediately.

At this point, I seldom bother with export lists, because I might change names around later on, and also I am working mostly in the package.

At the end of the hacking session, I can just evaluate (defpackage-form 'foo) to get the current state of package FOO conveniently as a DEFPACKAGE form, which I can then use to replace the initial stub. Eventually, if the code grows big enough to split it up, I move the package form to a separate file packages.lisp.

At least in the first iteration, the export list is likely still incomplete, so I can choose to include all symbols whose home package is the package I am working on, nicely sorted by name. Then I remove those symbols which are not meant for exporting. Like that, it is easy to bulk export many symbols at once. As a common convention, symbols starting with % are meant to be internal. They can be omitted from the export list automatically.

FORMAT Controls, the Ultimate Line Noise

Package forms print neatly (and by that I mean closer to how I would layout them manually) with the following additions:


(defun pprint-defpackage (stream defpackage-form)
  (format stream "~:<~W~^ ~3I~:_~W~^~1I~@{~:@_~:<~W~^ ~:I~@_~@{~W~^ ~_~}~:>~}~:>"
          defpackage-form))

(set-pprint-dispatch '(cons (member defpackage))
                     'pprint-defpackage)

Current versions of SBCL (newer than 1.0.17.36) come with this style included already. I also toyed with a two-column layout, but it is harder to edit.

Incremental Updates

Incremental updates of package forms are a little trickier, though. The generated form could, for example, be compared to the original with diff-sexp. I have not automated that stage much yet. If there are only few additions, it is probably easier to just add them manually.

Otherwise, I could imagine an Emacs before-save-hook which checks whether the export list of a package matches the DEFPACKAGE form in packages.lisp (or wherever it is stored—this could be figured out with source locations), and some more automation and integration with Emacs.

Very unfortunately, some loss of information occurs on the round trip from package form to object and back:

  • Reader conditionals inside DEFPACKAGE forms cannot be recreated easily. Neither can comments, for what it's worth.
  • There appears to be no way to figure out which package a symbol was actually imported from. For example, if we export CL:NIL from package FOO and then import FOO:NIL into some other package BAR, it will appear the same as if CL:NIL was imported directly into BAR.
  • The default :USE list (i.e., when none is specified) is implementation dependent, which naturally can cause some implementation dependent packages to show up in this list.
Lisp Logo (by Conrad Barsky)

Dozsa et al., "How Lisp Systems Look Different"

In this paper we propose a suite of new visualizations that reveal the special traits of the Lisp language and thus help in understanding complex Lisp systems. To validate our approach we apply them on several large Lisp case studies, and summarize our experience in terms of a series of recurring visual patterns that we have detected.

One of the case studies is SBCL.

Lisp Logo (by Conrad Barsky)

It amused me for while that I found myself writing something similar to the following piece of code:


(defun %apply (function arg &rest args)
  (apply #'apply function arg args))

(defun %funcall (function &rest args)
  (%apply function args))

I will leave it to you, dear reader, to figure out what the double APPLY is good for. Surprisingly, there is even some reason behind this madness, namely the ability to mangle the function argument before calling.

Lisp Logo (by Conrad Barsky)

There seems to be an interesting attraction between Erlang and Lisp and several times it has been tried to marry them, in different ways. Bill Clementson wrote about it already in his article Concurrent/Parallel Programming - The Next Generation. Here is an updated list:

  • ETOS, an Erlang to (Gambit) Scheme compiler, dating from 1997, was probably the first attempt to combine Erlang and Lisp. One of the things that came out of ETOS is Marc Feeley's paper A Case for the Unified Heap Approach to Erlang Memory Management.
  • Erlisp was another attempt, this time in Common Lisp. It's a (partial) reimplementation of the Erlang concurrency model. I don't think anybody is working actively on it at the moment.
  • CL-MUPROC is a Common Lisp library which strives to offer some of the multiprocessing abstractions found in Erlang, very much in the same way as Erlisp. It implements some of Erlang's fault-tolerance mechanisms, but not yet distributed operations.
  • Distel, a distributed Emacs Lisp with Erlang-style processes and message passing, and the Erlang distribution protocol, originally developed by Luke Gorrie of LtU, SLIME, OLPC, and other fame. Some time ago, Bill Clementson wrote a nice article about Distel.
  • Termite, another Erlang-like distributed programming system, this time written in Gambit-C Scheme.
  • Lisp Flavoured Erlang (LFE) is a Lisp syntax front-end to the Erlang compiler by Robert Virding (one of the inventors of Erlang). This is an interesting one, because it comes out of the Erlang community for a change, and instead of porting Erlang ideas to Lisp, it goes the other route: porting Lisp ideas to Erlang.
  • Erlang-in-Lisp, a Google LispNYC Summer of Code 2008 project, again very much in the spirit of Erlisp, it seems. The project is in its early stages, let's see how far this goes.
  • Erlang Server, Common Lisp Client by Berlin Brown appears to be the latest attempt of integrating Erlang and Lisp. They are connected via sockets and exchange messages in a custom protocol. This has the advantage that either side can easily be replaced by a different implementation.
  • erlang-scheme is a port of Distel from ELisp to Scheme.
  • Erlang-like Threads in Common Lisp

And this list does not even include projects like (again defunct?) Kali Scheme, which are clearly related.

Personally, I think that the reimplementation approach will have a tough stance against integration approaches like Distel. They lock out either one or the other of the two language eco-systems: libraries, development tools, etc., and recreating this is a lot of work (but don't let that stop you!) With Distel, I can choose to program parts of the application in ELisp or Erlang, whatever is a better fit.

Distel implements Erlang's on-the-wire protocol, which is nice because there is no need to mess around with a Foreign Function Interface. Alternatively, one could bind to the Erlang C libraries. As far as rapid prototyping is concerned, this should be the fastest and most straight-forward approach. I wonder why everybody is doing it the hard way (reimplementation)?

UPDATE 2008-07-06: Forgot ETOS

A helpful reader on reddit rightly pointed out that I forgot to include ETOS, now rectified.

UPDATE 2008-07-06: Erlang Interoperability

Bill Clementson wrote a nice summary of the Erlang Interoperability options. Thanks again, Bill!

UPDATE 2008-07-10: PLT Scheme meets Erlang

Grant Rettke pointed me to another Lisp-Erlang marriage: erlang-scheme

UPDATE 2008-12-17: Erlang Threads in Common Lisp

Jerome Baum announced an implementation of Erlang-like Threads in Common Lisp.

NetNewsWire: More news, less junk. Faster.

Reading news feeds? Forget Safari. Forget Apple Mail. Instead, try out NetNewsWire: It sports a convenient user interface and an integrated web browser (via WebKit), need I say more?

NetNewsWire Screenshot

I do not follow many news feeds, and after I switched to MacOS X, I used to use Safari's built-in feed reader. It was nothing to write home about, but it has a simple user interface and worked well enough for my modest needs. With OS X 10.5 Leopard, Apple Mail grew RSS support and I switched to it as default feed reader. I liked that it was the same interface as for email, but some things were rubbing me the wrong way. For instance, I created a Smart Mailbox to get rid of already read feed items: Smart Mailbox in Apple
Mail

The main show stopper was that some feeds basically require to visit a web page (e.g., the BBC News feed contains only a one-sentence summary of the stories. When in Apple Mail, this causes a switch to the default Web Browser, and possibly a desktop switch as well (thanks to Spaces).

With NetNewsWire, the awkward application switching (and backswitching) is history. Now, if only it could also view PDF files inline (via PDFKit)...

Page 9/29: « 5 6 7 8 9 10 11 12 13 »