1

I have read section 6.7 of LOL a few times now, and I still can't wrap my mind around the following.

Bindings that were previously closed to outside code are now wide open for us to tinker with, even if those bindings were compiled to something efficient and have long since had their accessor symbols forgotten.

If bound symbols are essentially compiled down to pointers in the environment of the closure, how can you pass a symbol to the already compiled function, and the function somehow is able to compare the symbol?

I've been messing with the pantest example in clisp, and I can see that I'm able to change both acc and this inside pantest. I can compile and disassemble pantest, but all the symbols show up in the environment. If I had a lisp that compiled down to assembly, I might gain some more intuition, but the code is complicated enough that it will probably be too difficult to follow without explanation.

Rainer Joswig
  • 136,269
  • 10
  • 221
  • 346
Todd
  • 475
  • 4
  • 15
  • 1
    I'd say that `pandoric-let` thing is much about whether they could without asking whether they should (and that order of evaluation...) But sure, you may add a case in `pandoric-let`'s `dlambda` that returns all variable names in it. – acelent Mar 06 '17 at 13:37

2 Answers2

2

I'm not familar with Let Over Lambda.

The book Lisp in Small Pieces explains how lexical binding can compile down to very efficient variable references. Since all known references to the variables are in a limited scope, you could use an array to store the bindings and reference them by a numeric index, rather than by using the symbol to look up things or using a property of the symbol to get the value.

A symbol passed into a function is just a symbol, a kind of data. Comparing it to other things in the function isn't the same as accessing information about the lexical bindings in a particular scope.

There's a didactic Lisp pseudo-OO technique that shows how you can change function behavior by passing in a symbol to access and modify the lexical state, but it's choosing from a fixed set of known things to compare, not an arbitrary lookup of lexical info based on a symbol.

(defun make-incrementor (initial-value)
  (let ((value initial-value))
    (lambda (action)
      (ecase action
        (:value
         value)
        (:increment
         (incf value))
        (:reset
         (setf value initial-value))))))

> (defvar *inc* (make-incrementor 10))
*INC*

> (funcall *inc* :increment)
11

> (funcall *inc* :increment)
12

> (funcall *inc* :increment)
13

> (funcall *inc* :reset)
10

This is manipulating the lexical binding of value without externally accessing it. All changes are mediated through code in the same lexical location.

Xach
  • 11,774
  • 37
  • 38
  • LOL also references Lisp in Small Pieces. I'll need to add it to my reading list. The point is, though, with Pandoric Macros, you can *externally* access value and any other unlimited number of symbols not yet even bound or defined in any other lexical environment (see plambda). This is what I don't understand. – Todd Mar 05 '17 at 16:41
  • Sounds like it may be a "once you know enough to do it, you will know enough not to do it" situation. – Xach Mar 05 '17 at 20:45
  • I'm not a lisp programmer. I think I started reading this book because I wanted to understand a little bit about how macros are used to make DSLs. Agreed that excess macro usage seems very dangerous. At the least, it's hard to understand what's going on. – Todd Mar 06 '17 at 19:56
1

(I will come back later and fill in some more information here later)

In short (and slightly oversimplified), the pandoric-let macro has added in some extra code to handle the case of trying to get or set each of the different variables it introduces. This extra code remembers the symbols of the variables after the code has been compiled, but since it's the only code which needs that information everything else gets compiled down to very efficient pointer operations.

The functions which generate this extra code are pandoriclet-get and pandoriclet-set, both of which are tricky to read because they return code which depends on the symbols sym and val which are provided in the pandoriclet macro itself.

djeis
  • 286
  • 2
  • 6
  • So are you saying the author's statement is not technically correct for Pandoric Macros? The symbols that are exported to other lexical environments are something more than just an index or pointer? – Todd Mar 07 '17 at 17:13
  • 1
    Well... I think I see the misconception. Lexical environments don't introduce symbols, they introduce variables and associate existing symbols with those variables. Normally, the compiler can see all the places where you referred to that variable by its symbol and it can optimize the symbol away entirely. What pandoric-let does is add some code to look up each variable by its symbol at runtime so you can access those variables outside the lexical scope. – djeis Mar 08 '17 at 17:57
  • I *think* I understand. Those symbols really are there (added in) in the closure after compilation. You still have to do some kind of comparison (I assume a string compare). The bindings the author refers to is presumably only the pointer to the variables and not the extra code compiled in. – Todd Mar 09 '17 at 00:56
  • 1
    Very close, however it's actually much simpler than that because of the lisp reader (`#'read`) which actually handles parsing the lisp code into lists-of-lists and generating the symbols. Whenever the reader comes across a symbol in the text that it's already seen, it just reuses the existing symbol object. That means (ignoring packages and gensyms for a moment) two symbols with the same name will always pointer/identity/`#'eq` compare as equivalent- no string comparison needed. In short, symbols are introduced long before compilation (`#'eval`). – djeis Mar 10 '17 at 17:28
  • But, I believe it's possible to load already compiled images. In this case, the reader could not have seen the symbol unless a text representation was stored somewhere; so, how could it compare #'eq? – Todd Mar 10 '17 at 23:32
  • The text/name of the symbol is stored as a field in the symbol object and in the package table. When the reader reads a new symbol it adds it to the package table by name, and that table is saved with the image. After reading, every symbol with the same name in the same package will be represented in memory by the same symbol object. Effectively, the string comparison happens once during reading instead of every time you compare two symbols. – djeis Mar 12 '17 at 00:29