There is no shortage of Lisp rants on the internet, sometimes entertaining, sometimes educational and sometimes both. In this post I’d like to dip my toes into those waters and offer my own mini-rant, centered on one particular idea about Lisp that is often repeated, usually in introductions to the language, which strikes me as not being very accurate. I want to talk about why this idea is actually true, in a sense, but not in the sense that it is usually put forward.
While there are many flavors of Lisp, any language that I would recognize as a Lisp is going to be largely the same in regard to the specific aspect addressed here, so in this post I will refer simply to “Lisp” as an abstraction over the set of actual Lisps. I will also avoid using Lisp-specific jargon (car, cdr, S-exp, etc.).
A Deepity
The philosopher Daniel Dennett has coined the term “deepity” to refer to an expression that has two interpretations: one that is seemingly profound but is not true, and one that is true but uninteresting. “Love is just a word,” you may hear someone say, and it will be clear after a moment of reflection that this is conflating the fact that there is a thing called love, whatever that thing may be, with the fact that there is a word for the thing called love.
When reading about Lisp, you often see it stated that “everything in Lisp is a list”. I would consider this to be something of a deepity. You can, if you want, consider a Lisp program as being made up entirely of lists and list items; you could also say that, because Java arrays can contain any type of object (putting aside the object/primitive distinction in that language), a Java program is composed entirely of arrays and array items. You could go on and say that Python programs are composed of nothing but dicts and dict keys and values. A car is a very simple machine when you consider that it is composed of only two parts: a steering wheel and the rest of it. Without belaboring the point further, it’s clear that while none of these ways of looking at things are wrong, strictly speaking, some familiarity with each subject shows them to be arbitrary distinctions that don’t do much to help us think about programs or cars. So why is this seen as a useful way to think about a Lisp program?
Programming languages are like butterflies in that they have to go through several phases before they reach their final form. Starting out as a string of characters in a source file, a program is tokenized, parsed into an abstract syntax tree, and if everything goes well, emerges from its cocoon as object code, or is executed by an interpreter as the case may be. This is as true for Lisp as for any language, but there is a difference in a Lisp program’s lifecycle, and that is the role of the Lisp reader, implemented by the built-in read function. This function creates Lisp objects, but rather than creating them from an application-specific serialization format, as with Python’s pickle format, it takes advantage of the fact that in Lisp everything is an expression, and so even the number 1 on its own, without any other context, can be understood as an expression which evaluates to itself. In fact, most values evaluate to themselves, with the few exceptions being lists and symbols.
This is where things get interesting, because unlike other languages, which consider the business of translating from source to object code to be a separate concern from programming in the language, Lisp instead uses one of its own build-in data types for representing Lisp source code. And that type is the list. Specifically, the list is what is used to represent a form to be evaluated, either as a function to apply to its arguments, a macro to expand, or a special form to evaluate according to its own definition. Symbols are likewise evaluated to the values bound to them. Because lists and symbols are perfectly good objects on their own, there are facilities for suppressing the evaluation that they would usually undergo with the quote and quasiquote (or backquote) special forms.
Literally Lisp
Most modern programming languages have a literal syntax—that is, a way to express a complex value with all of its components directly in code—for linear and associative arrays. The JSON format was created because of Douglas Crockford’s insight that the set of values for which JavaScript has a literal syntax is rich enough to serve as a general data exchange format, and it has been very successful in this capacity, but Lisp has the unique attribute that, thanks to read, the literal syntax for its linked-list type is identical to the syntax of the language itself. To put it another way, while many languages have string and array literals, Lisp has Lisp literals.
And now we can understand why people say that everything in Lisp is a list: when you are writing Lisp, you are using lists to structure your program, while atoms, understood simply as anything that is not a list, are what is being composed within that structure. The fact that code and data overlap visually causes the perception that all you are seeing is lists, lists everywhere. It’s almost an optical illusion. Am I looking at a duck or a rabbit? Am I looking at a linked list or a function application?
If this is all too abstract, a great way to get your head around it is to get a Lisp REPL going, using the Lisp of your choice, start typing things in and seeing what happens when you press return. And since the read function used by the REPL to parse your input is the same one that is available to you as a programmer, you can call read on different inputs as a way to experiment with this unique aspect of the language.
Lisp Mythbusting
And with that I’m going to end my contribution to the genre of Lisp rants, although there is much more to know about the Lisp reader works. Common Lisp, for example, has an extensible reader which allows you to define your own rules for how syntax is translated into runtime objects. Lisp’s simple, parentheses-based syntax is also what allows it to have macros, a language feature powerful enough that there have been entire books written about it. I will leave you with a quote from the late Erik Naggum, Lisp programmer and Usenet ranter, who expressed in a few sentences everything that I’ve been trying to say in this post (edited for standard capitalization and spacing instead of the idiosyncratic style Naggum used when posting to Usenet):
Its a persistent myth that Lisp has only one data type, the list. (…) I don’t think it’s wise to keep that myth alive.
Another persistent myth is that Lisp programs are lists. They aren’t. Lisp source code uses lists. Compiled, it is another data type, like a code vector, which includes native machine code or perhaps byte code.
What defines the Lisp family to me, after I have had the opportunity to refine my definition over time, is READ/WRITE CONSISTENCY. All languages in the extended Lisp family have this in common, and no language outside the Lisp family can sport anything like it. That is, the ability to print (write) some object out in text form and read it back in again (or the other way around, depending on where the object is originating).
Retrieved from https://www.xach.com/naggum/articles/3073836438931858@naggum.no.html, Mar. 28, 2021.
Leave a Reply