There are many gross characteristics of Common Lisp that make it significantly different from other common languages. Lisp is generally thought of as functional, and for the most part we shall adapt that view in this class. There are many aspects of Lisp that we shall barely touch upon, so the material you will see will contain many gross simplifications, and perhaps a few white lies. This can run you into trouble, and we the TA's shall be around to help you if this should happen.
I shall not comment on imperative programming beyond saying that C is a typical example. Functional programming implies a style of programming where functions are first class objects. In other words, they can be passed as arguments, created and used very flexibly. They are as much a data object as are integers. This is perhaps the single most confusing aspect of functional programming. Hopefully you will learn to appreciate functional programming as you gain more experience with it.
This command line embodies the read-eval-print loop. The command line reads an expression, evaluates it, and prints out a result. Part of the power of lisp comes from the fact that each of these functions is accessible to the programmer. (However, the use of eval is heavily deprecated for a variety of reasons.)
Some modern Common Lisp environments have a fancier graphical interface
available. The listener is however still the primary means for interacting
with the environment. The remaining components often include editors, inspectors,
debuggers, profilers, and a host of other tools. In over six years of working
with Lisp I have perhaps a couple of times asked for something more than
an emacs editor and the listener.
s-exp ::= atom | (symbol s-exp*) atom ::= symbol | number | string | <other things>In other words, an s-expression is an atom or a list expression. Each s-expression returns a value. The definition of an s-expression is recursive. Arbitrarily complex expressions can be created by composing s-expressions in this manner. This can however lead to programs that are hard to read, and is not advisable.
If the s-expression is a single atom, then the action the lisp environment takes depends on the type of atom. If it is a symbol, then the value of the symbol is accessed. Otherwise, atoms usually evaluate to themselves. (5, for example, returns 5 as its value.)
List s-expressions are quite straightforward too. The first component of these s-expressions is always a symbol. Evaluating the s-expression corresponds to executing the function associated with this symbol. The s-expressions following the symbol are also evaluated, and their values are used as arguments to the function. In the example below, the function associated with the symbol + is accessed. The atoms 2 and 3 are evaluated, and their return values (themselves) are used as arguments to the function +. The value of the s-expression is returned as the atom 5.
> (+ 2 3) 5Lisp, as you can see, uses prefix syntax.
fact(0) = 1 fact(n) = n*fact(n-1)Below are two methods for implementing factorial. In general, you will find that there are any number of ways of expressing an idea in lisp, and some of them turn out to be better than others.
(defun factorial (n) (if (= n 0) 1 (* n (factorial (1- n)))))This is the literal interpretation of the definition of factorial. Remember that each expression in Lisp returns a value, which allows for a simple translation of a variety of mathematical and logical constructs into equivalent programs.
The entire form is a single s-expression that constructs a function definition. The return value of this s-expression is the name of the function. DEFUN is a symbol in the function slot of the s-expression. It is not a function, but a special form. Unlike a function invocation, it does not necessarily evaluate all its arguments.
Like DEFUN, IF is a special form. It takes a variable number of s-expressions (two or three) in its argument slots. The first argument is necessarily evaluated. It is usually a predicate (a function that returns a boolean value). If the predicate returns true, then the second argument is evaluated. If false, the third argument is evaluated. The value of the entire IF s-expression is either the value of the second argument or the third, depending on which one is evaluated. If the third argument is absent, then NIL is returned.
The remaining s-expressions are mathematical functions that evaluate as you would expect them to. Let us run a few tests:
> (factorial 4) 24 > (factorial 30) 265252859812191058636308480000000Note on the last one that the result value was an exact integer. Common Lisp is limited only by memory in the size of the integers it can work with. You don't have to worry about overflows and such.
(defun factorial-1 (n product) (if (= n 0) product (factorial-1 (1- n) (* n product)))) (defun factorial (n) (factorial-1 n 1))At first glance, this version of FACTORIAL seems to be inferior to the first. It uses two functions, while the first uses only one. Version 2 is thus more verbose, and potentially more confusing. But this version has a very important advantage, for it allows a type of optimization called tail call elimination.
Consider the stack usage of version 1:
(factorial 3) => (* 3 (factorial 2)) => (* 3 (* 2 (factorial 1))) => (* 3 (* 2 (* 1 (factorial 0)))) => (* 3 (* 2 (* 1 1))) => (* 3 (* 2 1)) => (* 3 2) => 6Now let's look at version 2:
(factorial 3) => (factorial-1 3 1) => (factorial-1 2 3) => (factorial-1 1 6) => (factorial-1 0 6) => 6Version 1 requires the lisp environemnt to keep track of the partial results of the recursive calls, while version 2 keeps track of the partial result as an explicit argument. This allows the lisp environment to optimize version 2, by reusing the stack space of the previous call. Thus, version 2 can be easily reduced to a simple loop.
Of course, it is not always possible to reduce a recursive function
to a loop. Factorial involves linear recursion, since a call to the function
factorial results in at most one new call to factorial. In a tree recursive
algorithm, a call to the function can result in multiple new calls to the
function, which may make tail call elimination difficult or impossible.
We shall see an example of tree recursion with Towers of Hanoi, below.
A list in turn is composed of a series of cons cells. A cons cell is abstractly an object made of two slots. More concretely, it can be thought of as a two element array. Each of the slots can hold any kind of value, including other cons cells. Effectively, cons cells give us the ability to construct arbitrary tree structures.
Lists have a recursive definition.
> () NILThus, NIL is also the empty list.
> (cons 1 2) (1 . 2)The result of the above expression is called the dotted pair representation of a cons cell.
> (consp (cons 1 3)) T
> (list 1 2 3) (1 2 3)It automatially allocates and strings together all the necessary cons cells. The above expression is equivalent to
> (cons 1 (cons 2 (cons 3 ()))) (1 2 3)Note that the result contains no dotted pairs. That is because for readability the dotted pair representation is compressed to give a compact list representation. Generally, if the second component of a cons cell is another cons cell or (), the cons cell is not printed as a cons cell.
> (cons 1 (cons 2 3)) (1 2 . 3)The first cons cell is not printed as a dotted pair, while the second is.
> (car (cons 1 2)) 1
> (cdr (cons 1 2)) 2 > (cdr (list 1)) NIL
(caar x) <-> (car (car x)) (cadr x) <-> (car (cdr x))While convenient, excessive reliance on CAR and CDR can quickly make a program unreadable.
> (first (list 1 2 3)) 1 > (second (list 1 2 3)) 2
(nth 0 x) <-> (first x)
> (nthcdr 1 (list 1 2 3)) (2 3)
> (last (list 1 2 3)) (3)
> (listp ()) T > (consp ()) NIL
> (endp ()) T > (endp nil) TIn other words, ENDP performs precisely the same function as NULL. There are two different functions for the same purpose, since they communicate different information back to the reader. ENDP indicates that the argument is a list, while NULL does not. Consequently, ENDP should be used in preference to NULL when lists are involved.
A symbol is in fact an object with some internal structure. Two of these slots are of consequence to the programmer. They store the variable value and the function value of the symbol. In other words, a symbol can simultaneously have distinct variable and function values. The variable value of a symbol can be accessed by typing in the name of the symbol at the lisp prompt. For example,
> most-negative-single-float -3.4028238E38A symbol can be almost any string of characters. For example, +, A, A1, 1A, $ and $%* are all symbols. There are a few exceptions though, involving characters that play a special role in lisp syntax. Some of them are:
> (symbolp nil) T > (symbolp 4) NIL
As I have noted above, symbols can have variable values. T and NIL are special, in that they are constant objects that have themselves as variable values. In other words
> nil NIL > t TOne of their many functions is to act as booleans. NIL means false, and T means true. However, true is in actuality a lot more fuzzy, since anything not NIL is also true. This can be very handy, but can get rather confusing as additional meanings of NIL are introduced.
Exercise: What is (null (null t))?
(atom x) <=> (not (consp x)) > (atom 1) T > (atom (list 1)) NIL > (atom nil) T
As noted under the factorial example, numbers in lisp are not limited by the underlying machine architecture. They can be of arbitrary size, only limited by the available memory. All arithmetic operators function as one would expect on these numbers.
The function - requires at least one argument. If given only one argument, the return value is the argument's negation. If given more than one argument, the return value is the first argument minus the sum of the remaining arguments.
The function * without any arguments returns 1. With one argument the result is the argument itself. With more than one argument the product of all the arguments is returned.
The function / requires at least one argument. With one argument it returns the inverse of the argument. When more than one argument is supplied, the result is the first argument divided by the product of the remaining arguments.
1+ and 1- add 1 and subtract 1 from their arguments, respectively.
> (function +) #<Function +>The special form FUNCTION is hardly ever used. Special syntax provided in Common Lisp gives us a handy short cut.
> #'+ #<Function +>
> (functionp #'+) T
> (funcall #'+ 1 2 3) 6Thus, the above function call is equivalent to (+ 1 2 3). It is useful when a function has been passed as an argument, and cannot be executed in the usual manner.
> (apply #'+ (list 1 2 3)) 6 > (apply #'+ 4 (list 1 2 3)) 10All arguments other than the last are interpreted as individual arguments for the indicated function.
> "foo" "foo"
> (eql t t) T > (eql nil t) NIL > (eql 'a 'a) T > (eql 4 4) T > (eql (cons 1 2) (cons 1 2)) NIL > (let ((x (cons 1 2))) (eql x x)) T
> (equal (cons 1 2) (cons 1 2)) T
> (= 6 (+ 2 4) (* 2 3)) T
> (typep 5 'number) T > (typep nil 'list) T > (typep nil 'cons) NIL
(defun <function-name> <argument-list> <form1> <form2> ... <formN>)<function-name> has to be a valid symbol. (There are some exceptions that you should not concern yourself with just yet.) This symbol is returned as a result of the DEFUN expression. The <argument-list>, properly called the lambda list, is in its simplest form a list of symbols to which the arguments will be bound. When the function is executed, a new lexical context is created within which these bindings exist. It is in this context that <form1> through <formn> are evaluated, and the return value of the function is the result of the evaluation of <formn>.
We have already seen the FACTORIAL example and a sample use of a DEFUN expression.
> (funcall #'(lambda (x) (if (oddp x) (1+ x) x)) 5) 6As has been stated earlier, functions in Common Lisp are simply objects. We have seen two methods of creating functions so far. One is through DEFUN, which creates a named function, and the second is through LAMBDA, which creates an anonymous function. The main difference is that the symbol that names a function can be at the head of a complex s-expression, while an anonymous function must be invoked using FUNCALL or APPLY.
It is important to note the commonalities and differences between the two forms as well. The structure of the function definition is identical in both cases. A function definition requires the programmer to give a lambda (argument) list followed by a series of forms to evaluate. Both DEFUN and LAMBDA proceed to create function objects that behave identically when invoked. In fact, there are other special forms that create function objects. We shall not discuss them in the course of this tutorial.
> (quote x) XWithout the QUOTE, Common Lisp would attempt to access the variable value of the symbol. This form is used very often, to the point that it has a syntactic short-cut, the single quote.
> 'x XSymbols are not the only objects that may be quoted.
> '(1 2 3) (1 2 3) > '(1 . 2) (1 . 2) > (car '(1 . 2)) 1
(if <test> <then-form> <else-form>)On evaluation, the IF form first evaluates the <test> expression. If the value is non-NIL, the IF form returns the result of evaluating the <then-form>. Otherwise, the <else-form> is evaluated.
We have seen an example of the IF expression in FACTORIAL.
(cond (<test1> <form11> <form12> ... <form1K>) (<test2> form21> ... <form2L>) ... (<testN> <formN1> <formN2> ... <formNM>))The evaluation of a COND begins with the environment evaluating <test1>. If the test returns non-nil, then the forms following <test1> are evaluated, and COND returns the value of the last form under the test. If <test1> returns NIL, the environment proceeds to the next test. If all the tests are exhausted, then NIL is the return value of the COND.
A COND expression may have its last test (<testN> above) be T. This then is the default clause, whose forms are evaluated if all the other tests fail.
> (let ((x 50)) (cond ((oddp x) (1+ x)) ((> x 10) (/ x 2)) (t x))) 25
(when/unless <test> <form1> ... <formN>)For a WHEN expression, the <test> is evaluated. If it returns non-NIL, <form1> through <formN> are evaluated, and the return value of the WHEN expression is the return value of <formN>. UNLESS differs from WHEN in that it evaluates <form1>... <formN> when <test> returns NIL.
> (when t 5) 5 > (when nil 5) NIL > (unless t 5) NIL > (unless nil 5) 5
(let (<binding1>... <bindingN>) <form1>... <formN>)where each binding has the form
(<symbol> <value-expression>)The return value of the LET expression is the result of evaluating <formN>. It is instructive to compare LET with DEFUN. The symbols that LET binds are loosely equivalents of the arguments specified in a function definition. Like the arguments of a function, the values of the bindings are all evaluated in parallel. Then the forms that follow are evaluated, and the last form's value is returned.
LET* is different from LET in that it evaluates its bindings sequentially, so that the variable in <binding1> is available when the <value-expression> for <binding2>... <bindingN> are evaluated.
> (let ((x 2) (y 3)) (+ x y)) 5 > (let ((x 2) (y (* x 2))) (+ x y)) ERROR x is unbound ... > (let* ((x 2) (y (* x 2))) (+ x y)) 6 > (let ((x 2)) (let ((x 4) (y (* x 2))) (+ x y))) 8 > (let ((x 2)) (let* ((x 4) (y (* x 2))) (+ x y))) 12
> (or nil 5 t) 5
> (and nil 5 t) NIL > (and 5 t 'x) X
> (not 'x) NIL > (not nil) T
> (defvar *foo* 5) *FOO*The circumstances in which global variables are used are few, and excessive use can make programs difficult to read and debug. Use them sparingly.
(setq <symbol1> <value1> <symbol2> <value2> ... <symbolN> <valueN>)It does not establish a new lexical scope, but simply modifies the value of a variable within the current scope. It should be used as sparingly as possible. The return value of SETQ is the result of evaluating <valueN>.
> (let ((x 5)) (setq x (1+ x)) x) 6
> (let ((x (cons 1 2))) (setf (car x) 4) x) (4 . 2)
(let ((x 1) (y 2)) (+ x y))The LET form defines a lexical scope, within which variables X and Y are bound to 1 and 2, respectively. The expression (+ X Y) within this lexical scope accesses these bindings when it is evaluated. A lexical scope is roughly equivalent to a block in C.
Closures are one of the trickier topics for students new to lisp. A closure is simply a lexical environment within which a function has been defined. A closure then preserves all the bindings of the lexical environment for the function to use as needed. Consider the following:
> (let ((x 5)) (defun foo () x)) FOO > (foo) 5The value of the binding X is now completely inaccessible. It cannot be modified or accessed in any way, except by calling the function FOO. We have just constructed a lexical closure around FOO. Closures are rarely useful in this form, but there are a variety of other situations where they can be very powerful. Consider:
(defun make-foo-predicate (arg) (let ((processed-arg (some-complicated-processor arg))) #'(lambda (x) (foo-same-p x processed-arg))))This paradigm represents a method for constructing a closure that specifies the context for a customized predicate. This style of abstraction is often useful for providing abstractions otherwise difficult to provide.
Doing this right can be difficult. The ability to arbitrarily compose expressions is confusing to some, but is ultimately a great benefit when authoring a program.
Also, use the most specific possible function when you have a choice of forms to use. This is useful in many ways. It provides a form of self documenting code, in that the particular function used can tell the reader what data you had expected. When testing for end of list, it is better to use ENDP than NULL for this reason. NOT is preferable to NULL when you are expecting a boolean value. The other advantage is that you can catch errors at the right point, if you use a more restrictive function. For that reason it is better to use NTH than ELT if you want to refer to a particular element of a list.
Finally, you should start wondering about your abstractions if you find yourself using a lot of CAR and CDR and other list destructuring operations while accessing data.
Never use excessive whitespace. Compare:
(defun factorial-1 (n product) (if (= n 0) product (factorial-1 (1- n) (* n product))))with
( defun factorial-1 ( n product ) ( if ( = n 0 ) product ( factorial-1 ( 1- n ) ( * n product ) ) ) ; end if ) ; end defunThe latter for an experienced lisp programmer is really hard to read. The extra whitespace is confusing. The parentheses on lines by themselves even more so. In the first version, all you have to do is look at the indenting, and you can correctly identify the extent of each expression. In the second version, this process becomes very difficult.
A comment begins with a ; and extends for the remainder of the line. There are three generally accepted levels of comments. A single ; is an in-line comment, generally indicating a minor point relevant to a single line of code. Two ;; are used for comments inserted between lines of a function, and are for describing higher level issues within a function. Three ;;; are for comments between functions, and are for describing high level aspects of the program.
Finally, block comments are indicated by #| ... |# pairs. They are equivalent to /* ... */ in how they work, with one major advantage--they can be nested.
Each function is allowed a documentation string. For example,
(defun factorial-1 (n product) "Helper function for calculating the factorial of a number." (if (= n 0) product (factorial-1 (1- n) (* n product))))This documentation string immediately follows the function lambda list, and should concisely describe the function. For major top-level functions the documentation string is often much more elaborate.
WHEN and UNLESS indicate that you only care about the consequent case, and not the alternate. IF on the other hand indicates that you care about both the consequent and the alternate cases. IF however is restricted to a single expression for the alternate and the consequent. A programmer that requires either of these to be multiple lines, or requires to specify else-if cases, should consider using COND.
(let ((x ...) (y ...)) (let ((z (f x))) ...))Consider switching to LET* in this situation. Your code would then reduce to
(let* ((x ...) (y ...) (z (f x))) ...)
The problem is conceptually fairly simple to solve. To move n rings from the start to the destination, move n-1 from the start to the spare, then move the nth ring from the start to the destination. Finally, move the n-1 rings on the spare to the destination, using the start peg as the spare. Our task is to write a function that tells us what moves to make to get all n rings from the start peg to the destination peg. From this description alone, it is possible to deduce that the most natural way of expressing the solution is as a recursive function, and that the recursion will be tree recursion rather than linear recursion.
In typical lisp style, we want to write a function TOWERS-OF-HANOI which takes one argument, the number of rings on the start peg. Our return value should be a list of moves. Given the nature of the problem, a very simple system suffices for describing the move. We can only move one ring, the top one, at a time, so it is sufficient to state the source peg and the destination peg to fully describe a move. A cons cell is ideally suited for storing such a pair. We thus have a data input and output specification in lisp terms, and a conceptually straight-forward algorithm. This yields the following implementation:
(defun toh (height start-peg-name extra-peg-name end-peg-name) (cond ((= height 0) nil) ((= height 1) (list (cons start-peg-name end-peg-name))) (t (append (toh (1- height) start-peg-name end-peg-name extra-peg-name) (toh 1 start-peg-name extra-peg-name end-peg-name) (toh (1- height) extra-peg-name start-peg-name end-peg-name))))) (defun towers-of-hanoi (height) (toh height 'start-peg 'extra-peg 'end-peg))There are only a handful of issues to point out here. Note the use of COND rather than nested IF statements. COND is preferentially used when such nesting is necessary. The other is the consistent application of the choice of representation. Every return value that TOH can possibly return is a list, so it is trivial to apply the APPEND operator to the results of the different recursive steps. As you might guess, APPEND sticks together the contents of all its argument lists into a single list. It allocates new cons cells as necessary.
Exercise: Try writing a definition for APPEND.
Sample runs (formatted for neatness):
> (tower-of-hanoi 3) ((START-PEG . END-PEG) (START-PEG . EXTRA-PEG) (END-PEG . EXTRA-PEG) (START-PEG . END-PEG) (EXTRA-PEG . START-PEG) (EXTRA-PEG . END-PEG) (START-PEG . END-PEG)) > (tower-of-hanoi 4) ((START-PEG . EXTRA-PEG) (START-PEG . END-PEG) (EXTRA-PEG . END-PEG) (START-PEG . EXTRA-PEG) (END-PEG . START-PEG) (END-PEG . EXTRA-PEG) (START-PEG . EXTRA-PEG) (START-PEG . END-PEG) (EXTRA-PEG . END-PEG) (EXTRA-PEG . START-PEG) (END-PEG . START-PEG) (EXTRA-PEG . END-PEG) (START-PEG . EXTRA-PEG) (START-PEG . END-PEG) (EXTRA-PEG . END-PEG))Note the exponential growth of the size of the solution. I had read once that Vietnamese tradition had it that the world would end when a puzzle of 64 rings, each made of stone, was solved. My apologies if this is a bit of urban legend I happened to pick up. The analysis that followed demonstrated that such a feat would take an impossibly long time. Something like half the age of the universe if a ring was moved every second.
We shall use the tools we have picked up so far for the task. The only data structure we have to represent a sequence of objects is a list, so our choice is rather limited. (I would not recommend that anyone use lists to represent matrices though. Lisp has multi-dimensional arrays.) We will use a row-major representation, thus:
Empty matrix => nil m by n matrix => ((x11 x12 ... x1n) (x21 x22 ... x2n) . . (xm1 xm2 ... xmn))The transpose problem can be viewed as follows:
First, consider adding a column to an empty matrix. The following transformation can serve as an example:
(add-column (list 1 2) nil) => ((1) (2))Each row has exactly one element, and the number of rows is the same as the length of the column. Looking at this operation from the perspective of list operations, we have simply enclosed each element of the column in its own list. This is called a mapping operation, where the same transformation is applied to all the elements of a collection. The functional style provides convenient abstractions for performing such mappings, through the function MAPCAR.
Adding a column to an existing matrix is similarly easy. Conceptually, we want to add each element of the column to the corresponding row in the matrix. That is, we want a mapping from the column and existing matrix to a new matrix that combines the two.
Here is the final implementation:
(defun empty-matrix-p (m) (null m)) (defun add-column (column m) (if (empty-matrix-p m) (mapcar #'list column) (mapcar #'cons column m))) (defun matrix-transpose (m) (if (empty-matrix-p m) m (add-column (car m) (matrix-transpose (cdr m)))))Note the definition of EMPTY-MATRIX-P. It simply checks if its argument is NIL. It is generally inadvisable to write functions that do so little, unless it is for providing an appropriate abstraction. The advantage in creating such abstractions is that the underlying representation can be modified, without having to worry very much about the effect on the remainder of the program. This level of abstraction has not been used uniformally in MATRIX-TRANSPOSE, which is a potential problem.
Some examples, again formatted for legibility:
> (matrix-transpose nil) NIL > (matrix-transpose (list (list 1 2) (list 3 4))) ((1 3) (2 4))
Below is the implementation of this simple linear recursive function:
(defun my-mapcar (fn list) (if (endp list) nil (cons (funcall fn (car list)) (my-mapcar fn (cdr list)))))
(defun my-mapcar (fn first-l &rest rest-l) (if (or (endp first-l) (some #'endp rest-l)) nil (cons (apply fn (car first-l) (my-mapcar #'car rest-l)) (apply #'my-mapcar fn (cdr first-l) (my-mapcar #'cdr rest-l)))))This is the first function we have written that takes multiple arguments, so it deserves special attention. Note the &REST in the lambda list of MY-MAPCAR. &REST is a special keyword that instructs the environment to collect the remaining arguments supplied to a function into a list, which is bound to the supplied name. A simpler example is given below:
> (defun my-list (&rest l) l) MY-LIST > (my-list 1 2 3) (1 2 3)This function replicates the functionality of the function LIST, though this is not the recommended implementation. Note that individual items supplied to MY-LIST are collected into a list.
Let us return now to MY-MAPCAR. Note that we require at least one list to be given as an argument, so we have both a regular list argument and an &REST argument.
The next step is to implement the predicate for the base case. Testing for the end of a single list is the same as it was for the single list case. The predicate ENDP gives us what we need. We use the function SOME to test for the end of a list of lists. SOME is a specialized mapping function, that tests if the predicate returns non-NIL for some element of the argument list. (It too can take multiple lists as arguments.)
Exercise: Look up the definition of SOME. Try to implement it.
The recursive test is again complicated by the existence of multiple lists. Now that we have established that none of the lists is empty, we want to obtain the result of calling the function with the first element of each of the lists. The first element of a single list can be trivially obtained through CAR. Obtaining the first element of a collection of lists is again a mapping operation, for which we again use MY-MAPCAR, where we fetch the CAR of each element in the collection of lists. We then invoke the mapping function using APPLY, since it can take a list of values as arguments to apply, while FUNCALL cannot.
The recursive call to MY-MAPCAR again involves a similar set of collection
functions. Make sure you understand how it works.
Last modified: Tue Apr 6 16:59:40 EDT 1999