How to use libraries in Common Lisp?

Question

I'm a beginner in Common Lisp and I want to use a library.

I can't find a single one simple example of loading / requiring / using a module. I've installed cl-ppcre like this :

$ sbcl --non-interactive --eval '(ql:quickload "cl-ppcre")'
To load "cl-ppcre":
  Load 1 ASDF system:
    cl-ppcre
; Loading "cl-ppcre"
..

But I don't know how to actually use it. I've tried the following and a dozen other thing and not one works.

$ sbcl --noinform --non-interactive --eval '(progn (require "cl-ppcre") (cl-ppcre:split "\s+" "1 2 3"))'
Unhandled SB-INT:SIMPLE-READER-PACKAGE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING
                                                          {1004DB8073}>:
  Package CL-PPCRE does not exist.

    Stream: #<dynamic-extent STRING-INPUT-STREAM (unavailable) from "(progn (...">

Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {1004DB8073}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<SB-INT:SIMPLE-READER-PACKAGE-ERROR "Package ~A does not exist." {1003640A83}> #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<SB-INT:SIMPLE-READER-PACKAGE-ERROR "Package ~A does not exist." {1003640A83}>)
2: (INVOKE-DEBUGGER #<SB-INT:SIMPLE-READER-PACKAGE-ERROR "Package ~A does not exist." {1003640A83}>)
3: (ERROR #<SB-INT:SIMPLE-READER-PACKAGE-ERROR "Package ~A does not exist." {1003640A83}>)

So how can I make it work ?

EDIT 1: I didn't precised that my problem with using libraries is as much a in scripts as in the terminal. It was implicit to me. This is because of my experience in Perl, in which everything you can do with a file, you can do at the command line, including using libraries.

EDIT 2: Here is my working solution. As it turned out, there were 2 things were wrong. My problem required to :

1. using multiple --eval

Just like Svante and ignis volens said.

2. (load "~/.quicklisp/setup.lisp")

Which I was alrady explained here :

Confused about ``ql:quickload`` and executable scripts in SBCL

This is the terminal solution :

sbcl --non-interactive --eval '(load "~/.quicklisp/setup.lisp")' --eval '(require :cl-ppcre)' --eval '(princ (cl-ppcre:split "\\s+" "1  2 3"))'

With the caveat that a bunch of warnings are outputed on stderr, like this one, and I don't know why that is.

WARNING: redefining QL-SETUP:QMERGE in DEFUN

And this is the script solution :

#!/usr/bin/sbcl --script

(load "~/.quicklisp/setup.lisp")
(require :cl-ppcre)

(princ (cl-ppcre:split "\\s+" "1    2 3"))
(terpri)

Answer 1

This is an instance of a common problem people have with CL.

The way CL (and other Lisps) work is in three phases (in fact more than three, but three is enough):

1. a sequence of characters is read to turn it into a form;
2. various magic happens to that form;
3. the result of stage 2 is evaluated, and perhaps the results printed.

Typically this process is iterated to process, say, a file, or a stream of input.

The important thing is that (1), (2), and (3) happen in sequence: (1) is complete before (2) begins, and both (1) and (2) are complete before (3) begins.

What that means is that (1) must be possible before anything that happens in (2) or (3) have happened.

So consider this form:

(progn 
  (ql:quickload "cl-ppcre")
  (cl-ppcre:split "\s+" "1 2 3"))

(This is pretty much one of the forms you are trying to evaluate.)

So the question is: what does it take for (1) to happen? Well, it takes two things:

• the QL package (which is essentially a namespace: see below for more on what 'package' means in CL) must exist to read ql:quickload;
• the CL-PPCRE package must exist to read cl-ppcre:split.

And now you see the problem: (ql:quickload "cl-ppcre") creates the CL-PPCRE package, and this does not take place until (3). That means that this form can't be read.

You can get around this, in desperation, with various heroic tricks. However you don't actually need to: you can do something else, which (almost: see below) works:

(ql:quickload "cl-ppcre")
(cl-ppcre:split "\s+" "1 2 3")

And this is (almost) fine, because it's not one form: it's two. So (1)-(3) work fine for the first form, and then (1)-(3) work fine for the second form.

So the answer is not to try and bundle everything into a single form. Either put things in a file (probably the best way) or if you really want to run things as command line arguments, you need to arrange that the forms are separate, with, for instance, multiple --eval options.

---

Above I said that the second, multiple-form, version almost works. And it does only almost work. The reason for this is file compilation. Let's assume I have a file whose contents are:

(ql:quickload "cl-ppcre")
(cl-ppcre:scan ...)
...

Well, what does the compiler do when it compiles that file? It reads it, form by form, but in general it doesn't actually execute code (there are exceptions): it arranges for that code to be executed when the file is loaded. So the compiler will not load CL-PPCRE: it will arrange life so that when the file is loaded CL-PPCRE will be loaded. And now we have a version of the same problem: the second form can't be read by the compiler because the CL-PPCRE package does not yet exist.

Well, there is a solution to that: you can tell the compiler that it must, in fact, execute some code at compile-time:

(eval-when (:compile-toplevel :load-toplevel :execute)
  (ql:quickload "cl-ppcre"))
(cl-ppcre:scan ...)
...

And now the compiler knows, thanks to the eval-when form, that it must call ql:quickload. And it will do so, and so the CL-PPCRE package will be defined at compile time, and all will be well.

---

A note on packages in CL

The term 'package' in CL has a meaning which is unfortunately not the same as it is in many other languages: that's because of history and can't now be changed.

In common usage, a package is 'some chunk of code which you can perhaps install and which perhaps has dependencies on other packages (chunks of code), all of which might be looked after by a package manager of some kind. You might install Python as a package on your Ubuntu box, or you might use the conda package manager to manage scientific Python packages (including Python itself).

In CL a package is essentially a namespace. If I type 'foo:bar' then this is referring to the symbol named BAR available in the package one of whose names or nicknames is FOO. Further this is an 'external' symbol, which means it's intended to be public in some way. Packages are real objects in CL and can be reasoned about by programs. There is always a notion of a current package, and that package defines what names are available without requiring package prefixes both by directly containing some names and also having a search ('use') list of other packages. There is a lot to packages in CL: far more than I can mention here.

What commonly might be referred to as packages are probably best referred to as 'libraries' in CL: they're chunks of code which you can install and which may have dependencies in their own right. Alternatively they are often referred to as 'systems' as they are often defined using a 'system definition' tool. 'Library' is probably the better term: 'system' is again a historical oddity which can't really be changed now.

Answer 2

Running programs from bash parameters as source is not the simplest way to start. The problem is the order of compilation, loading, and executing. Roughly, the parts you enter on the command line are first compiled, then loaded, but it would need to load the library first before it can compile the rest. Thre might be ways around this, e. g. with multiple -e parameters, but I don't think that this is a useful exercise for a beginner.

If you just want to try things out from a command line, start SBCL without parameters, which will give you a REPL prompt. That's a command line which accepts Lisp code. It looks like * in vanilla SBCL.

[user@home]> sbcl
This is SBCL …
*

At that prompt, load your library:

* (ql:quickload "cl-ppcre")
To load "cl-ppcre":
  Load 1 ASDF system:
    cl-ppcre
; Loading "cl-ppcre"
[package cl-ppcre]................................
..........................
("cl-ppcre")
* 

Then use it:

* (cl-ppcre:split "\s+" "1 2 3")
("1 2 3")
* 

Then you can quit the REPL:

* (quit)
[user@home]>

For serious work, use files.

Answer 3

Packages

Definitions

• a library or application is called a system.
• packages are namespaces for symbols
• symbols are named objects which often reside in a package

FOO::BAR denotes the symbol named BAR in the package FOO

FOO:BAR denotes the exported symbol named BAR in the package FOO

BAR denotes the symbol BAR in the current package

To read a symbol with a specific package, that package needs to exist

Example:

In LispWorks there is a package called CAPI -> we can find it.

CL-USER 44 > (find-package "CAPI")
#<The CAPI package, 5109/8192 internal, 880/1024 external>

CL-USER 45 > (read-from-string "CAPI::BAR")
CAPI::BAR
9

above creates a symbol named BAR in the existing package CAPI.

But there is no package called FOO. We can't find it:

CL-USER 46 > (find-package "FOO")
NIL

Now reading a symbol which is in package FOO is an error, since the package does not exist:

CL-USER 47 > (read-from-string "FOO::BAR")

Error: Reader cannot find package FOO.
  1 (continue) Create the FOO package.
  2 Use another package instead of FOO.
  3 Try finding package FOO again.
  4 (abort) Return to top loop level 0.

Type :b for backtrace or :c <option number> to proceed.
Type :bug-form "<subject>" for a bug report template or :? for other options.

CL-USER 48 : 1 > 

Problem

When you have one Lisp form, you need to make sure that it does not contain unknown packages. So first load the system with the software, which also defines the packages. Then find the symbol cl-ppcre:split by name split in the package cl-ppcre:

(progn
  (ql:quickload "cl-ppcre")
  (funcall (find-symbol "SPLIT" "CL-PPCRE") "\\s+" "1 2 3"))

Above avoids mentioning the symbol cl-ppcre:split. Instead we are looking it up at runtime via find-symbol.

Or:

(progn
   (ql:quickload "cl-ppcre")
   (eval (read-from-string "(cl-ppcre:split  \"\\\\s+\" \"1 2 3\")")))

Or: make it two forms:

  (ql:quickload "cl-ppcre")  ; loading the software
                             ;  also creates the package CL-PPCRE
  (cl-ppcre:split "CL-PPCRE") "\\s+" "1 2 3"))

Answer 4

You're almost there.

You can use (ql:quickload package) to install and use. If it's not on your system, it will be downloaded.

Unless you also import the package, you have to prefix the function with the library name.

Can you make this work in your development system?

 (ql:quickload :arrow-macros)

 (defun do-it ()
     (arrow-macros:->>
         '(1 2 3 4)
          (mapcar #'1+)
          (reduce #'+)
          ))

Answer 5

There is an example of how to do this on the quicklisp page itself: https://www.quicklisp.org/beta/

The example installs quicklisp and eventually uses a library "vecto".