69

I've encounter a following statement by Richard Stallman:

'When you start a Lisp system, it enters a read-eval-print loop. Most other languages have nothing comparable to read, nothing comparable to eval, and nothing comparable to print. What gaping deficiencies! '

Now, I did very little programming in Lisp, but I've wrote considerable amount of code in Python and recently a little in Erlang. My impression was that these languages also offer read-eval-print loop, but Stallman disagrees (at least about Python):

'I skimmed documentation of Python after people told me it was fundamentally similar to Lisp. My conclusion is that that is not so. When you start Lisp, it does 'read', 'eval', and 'print', all of which are missing in Python.'

Is there really a fundamental technical difference between Lisp's and Python's read-eval-print loops? Can you give examples of things that Lisp REPL makes easy and that are difficult to do in Python?

James McMahon
  • 48,506
  • 64
  • 207
  • 283
Jan Wrobel
  • 6,969
  • 3
  • 37
  • 53

4 Answers4

64

In support of Stallman's position, Python does not do the same thing as typical Lisp systems in the following areas:

  • The read function in Lisp reads an S-expression, which represents an arbitrary data structure that can either be treated as data, or evaluated as code. The closest thing in Python reads a single string, which you would have to parse yourself if you want it to mean anything.

  • The eval function in Lisp can execute any Lisp code. The eval function in Python evaluates only expressions, and needs the exec statement to run statements. But both these work with Python source code represented as text, and you have to jump through a bunch of hoops to "eval" a Python AST.

  • The print function in Lisp writes out an S-expression in exactly the same form that read accepts. print in Python prints out something defined by the data you're trying to print, which is certainly not always reversible.

Stallman's statement is a bit disingenuous, because clearly Python does have functions named exactly eval and print, but they do something different (and inferior) to what he expects.

In my opinion, Python does have some aspects similar to Lisp, and I can understand why people might have recommended that Stallman look into Python. However, as Paul Graham argues in What Made Lisp Different, any programming language that includes all the capabilities of Lisp, must also be Lisp.

Greg Hewgill
  • 951,095
  • 183
  • 1,149
  • 1,285
  • 2
    Indeed, RMS may have preferred for `print()` to use `repr()` instead of `str()`. That said, `print(repr(eval(raw_input("> "))))` is pretty close to a REPL. – Frédéric Hamidi Sep 03 '12 at 19:49
  • Note that `eval` and `exec` also work with code objects (returned by `compile`), not only strings. – user4815162342 Sep 03 '12 at 19:54
  • 1
    @user4815162342: Yes, that's what I meant by "jump through a bunch of hoops". – Greg Hewgill Sep 03 '12 at 19:55
  • 11
    Python makes a distinction between data and code. LISP doesn't. See also [Greenspun's tenth rule](http://en.wikipedia.org/wiki/Greenspun's_tenth_rule) – Henk Langeveld Sep 03 '12 at 19:55
  • A single call to `compile` is hardly a bunch of hoops. Stallman's point is that Python is missing the *concept* of the reader and the flexibility it provides in handling of data-as-code, not that Python makes it hard to provide `eval` with something other than a string. – user4815162342 Sep 03 '12 at 20:02
  • 3
    @HenkLangeveld Does it though? Python has first-class code representations; lisp takes textual input in the form of sequences of characters. – Marcin Sep 03 '12 at 21:04
  • @HenkLangeveld, not all Lisps are equal. See the syntax object definition in R6RS. – SK-logic Sep 05 '12 at 14:04
  • 2
    But the python interactive prompt doesn't read "a single string". It reads a complete expression (often over several lines), which can evaluate to data or code (an expression or a statement). Since functions are first-class objects in python, the equivalent of `eval` is to simply run the object: `name()`, if `name` refers to a function. Only `print` does indeed have different properties: printing a python expression or function does not usually give us something that can be parsed the same way again. – alexis Sep 07 '12 at 17:07
  • Stallman is known for having these kinds of... high standards. – Karl Knechtel May 30 '22 at 00:55
33

Stallman's point is that not implementing an explicit "reader" makes Python's REPL appear crippled compared to Lisps because it removes a crucial step from the REPL process. Reader is the component that transforms a textual input stream into the memory — think of something like an XML parser built into the language and used for both source code and for data. This is useful not only for writing macros (which would in theory be possible in Python with the ast module), but also for debugging and introspection.

Say you're interested in how the incf special form is implemented. You can test it like this:

[4]> (macroexpand '(incf a))
(SETQ A (+ A 1)) ;

But incf can do much more than incrementing symbol values. What exactly does it do when asked to increment a hash table entry? Let's see:

[2]> (macroexpand '(incf (gethash htable key)))
(LET* ((#:G3069 HTABLE) (#:G3070 KEY) (#:G3071 (+ (GETHASH #:G3069 #:G3070) 1)))
 (SYSTEM::PUTHASH #:G3069 #:G3070 #:G3071)) ;

Here we learn that incf calls a system-specific puthash function, which is an implementation detail of this Common Lisp system. Note how the "printer" is making use of features known to the "reader", such as introducing anonymous symbols with the #: syntax, and referring to the same symbols within the scope of the expanded expression. Emulating this kind of inspection in Python would be much more verbose and less accessible.

In addition to the obvious uses at the REPL, experienced Lispers use print and read in the code as a simple and readily available serialization tool, comparable to XML or json. While Python has the str function, equivalent to Lisp's print, it lacks the equivalent of read, the closest equivalent being eval. eval of course conflates two different concepts, parsing and evaluation, which leads to problems like this and solutions like this and is a recurring topic on Python forums. This would not be an issue in Lisp precisely because the reader and the evaluator are cleanly separated.

Finally, advanced features of the reader facility enable the programmer to extend the language in ways that even macros could not otherwise provide. A perfect example of such making hard things possible is the infix package by Mark Kantrowitz, implementing a full-featured infix syntax as a reader macro.

Community
  • 1
  • 1
user4815162342
  • 141,790
  • 18
  • 296
  • 355
22

In a Lisp-based system one typically develops the program while it is running from the REPL (read eval print loop). So it integrates a bunch of tools: completion, editor, command-line-interpreter, debugger, ... The default is to have that. Type an expression with an error - you are in another REPL level with some debugging commands enabled. You actually have to do something to get rid of this behavior.

You can have two different meanings of the REPL concept:

  • the Read Eval Print Loop like in Lisp (or a few other similar languages). It reads programs and data, it evaluates and prints the result data. Python does not work this way. Lisp's REPL allows you to work directly in a meta-programming way, writing code which generates (code), check the expansions, transform actual code, etc.. Lisp has read/eval/print as the top loop. Python has something like readstring/evaluate/printstring as the top-loop.

  • the Command Line Interface. An interactive shell. See for example for IPython. Compare that to Common Lisp's SLIME.

The default shell of Python in default mode is not really that powerful for interactive use:

Python 2.7.2 (default, Jun 20 2012, 16:23:33) 
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> a+2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'a' is not defined
>>>

You get an error message and that's it.

Compare that to the CLISP REPL:

rjmba:~ joswig$ clisp
  i i i i i i i       ooooo    o        ooooooo   ooooo   ooooo
  I I I I I I I      8     8   8           8     8     o  8    8
  I  \ `+' /  I      8         8           8     8        8    8
   \  `-+-'  /       8         8           8      ooooo   8oooo
    `-__|__-'        8         8           8           8  8
        |            8     o   8           8     o     8  8
  ------+------       ooooo    8oooooo  ooo8ooo   ooooo   8

Welcome to GNU CLISP 2.49 (2010-07-07) <http://clisp.cons.org/>

Copyright (c) Bruno Haible, Michael Stoll 1992, 1993
Copyright (c) Bruno Haible, Marcus Daniels 1994-1997
Copyright (c) Bruno Haible, Pierpaolo Bernardi, Sam Steingold 1998
Copyright (c) Bruno Haible, Sam Steingold 1999-2000
Copyright (c) Sam Steingold, Bruno Haible 2001-2010

Type :h and hit Enter for context help.

[1]> (+ a 2)

*** - SYSTEM::READ-EVAL-PRINT: variable A has no value
The following restarts are available:
USE-VALUE      :R1      Input a value to be used instead of A.
STORE-VALUE    :R2      Input a new value for A.
ABORT          :R3      Abort main loop
Break 1 [2]>

CLISP uses Lisp's condition system to break into a debugger REPL. It presents some restarts. Within the error context, the new REPL provides extended commands.

Let's use the :R1 restart:

Break 1 [2]> :r1
Use instead of A> 2
4
[3]>

Thus you get interactive repair of programs and execution runs...

Rainer Joswig
  • 136,269
  • 10
  • 221
  • 346
  • 4
    Yes, but the thing about python is that the interpreter prompt *does* facilitate interactive development. It really sounds like RMS looked at python, concluded correctly that it is not in fact lisp, and declared it inferior. – Marcin Sep 03 '12 at 20:14
  • 5
    @Marcin, it does. But the default 'interpreter' prompt is not very powerful for interactive development. Just as a CLI, not as a REPL which reads/evals/prints. Stallman was coming from a world where the interactive prompt had much more functionality. Including his own Emacs. – Rainer Joswig Sep 03 '12 at 20:16
  • 3
    In practice, in my own experience, I would not say that the Common Lisp repl is more helpful by itself. – Marcin Sep 03 '12 at 20:22
  • @Marcin: Common Lisp is a language. The REPL is barely defined in the language spec. Which REPL of which implementation do you think of? – Rainer Joswig Sep 03 '12 at 20:22
  • 3
    I'm thinking of the CLISP, CMUCL, and SBCL repls. Comparing IPython to SLIME is like comparing sed to Emacs. They are not even close to being the same thing, and in any case, neither is the subject of this question. – Marcin Sep 03 '12 at 20:29
  • 3
    @Marcin: My impression is that the CLISP REPL is quite a bit more powerful than what Python provides by default. – Rainer Joswig Sep 03 '12 at 20:39
  • Whether or not it is "more powerful" my experience has not been that it is more useful for interactive development; if anything, the reverse - the additional features interrupt the flow of interaction. – Marcin Sep 03 '12 at 20:58
  • @Marcin: A bare REPL isn't that usefull, unless you're missing SLIME, but with SLIME it's awesome. – Daimrod Sep 03 '12 at 21:16
  • 2
    In addition, it is possible to access the origin of an error in the python interpreter by doing `import pdb; pdb.pm()`. The difference is that python doesn't automatically take you to the debugger by default (which I, for one, prefer). – Marcin Sep 17 '12 at 18:28
  • It's possible to get to the python debugger automatically if you start ipython like this: `ipython --pdb`. While I think @RainerJoswig is right, his particular example `(+ a 2)` is not very illustrative IMO. – aadcg Jun 19 '21 at 10:00
7

Python's interactive mode differs from Python's "read code from file" mode in several, small, crucial ways, probably inherent in the textual representation of the language. Python is also not homoiconic, something that makes me call it "interactive mode" rather than "read-eval-print loop". That aside, I'd say that it is more a difference of grade than a difference in kind.

Now, something tahtactually comes close to "difference in kind", in a Python code file, you can easily insert blank lines:

def foo(n):
  m = n + 1

  return m

If you try to paste the identical code into the interpreter, it will consider the function to be "closed" and complain that you have a naked return statement at the wrong indentation. This does not happen in (Common) Lisp.

Furthermore, there are some rather handy convenience variables in Common Lisp (CL) that are not available (at least as far as I know) in Python. Both CL and Python have "value of last expression" (* in CL, _ in Python), but CL also has ** (value of expression before last) and *** (the value of the one before that) and +, ++ and +++ (the expressions themselves). CL also doesn't distinguish between expressions and statements (in essence, everything is an expression) and all of that does help build a much richer REPL experience.

As I said at the beginning, it is more a difference in grade than difference in kind. But had the gap been only a smidgen wider between them, it would probably be a difference in kind, as well.

Vatine
  • 20,782
  • 4
  • 54
  • 70
  • 1
    In what way is homoiconicity implicit in the term "read-eval-print loop"? – Marcin Sep 17 '12 at 18:25
  • 2
    @Marcin It is not a strict requirement, but the only time I have heard the term read-eval-print loop has been with homoiconic languages, the rest of them tend to refer to "interactive mode" or "the interpreter" (essentially, if Python has a REPL, so does Sinclair Basic) – Vatine Sep 18 '12 at 10:20
  • Yes, I think it is fair to say that Sinclair Basic has a REPL. – Marcin Sep 18 '12 at 12:19
  • 1
    Sinclair Basic's interactive prompt is no REPL as it misses the print part. It prints only what you order it to print, and what it prints cannot generally be read back. – Marko Topolnik Sep 19 '12 at 08:18
  • 1
    @MarkoTopolnik: In that case, neither is Python (in Sinclair Basic, "3+4" is not a valid statement (it is, in Python and causes 7 to be written), "LET I=3+4" doesn't print anything and neither does "i=3+4" in Python; the closest Sinclair Basic gets is "PRINT 3+4" and that does, as happens, print 7). – Vatine Oct 10 '12 at 10:34
  • 2
    @Vatine Yes, that's the point we're discussing here: Python's interactive prompt is not a REPL. Also note that Sinclair's prompt is even further away from the REPL: you cannot reuse **anything** it prints. Even the concept of a TTY is missing, where the output history is retained, as if on a continuous-feed printer (the original TTY). – Marko Topolnik Oct 10 '12 at 10:37