In elisp, how do I apply backquote to lists read from files

Question

I would like to take a large list (think faces in an emacs theme) and break it up into smaller lists in separate files. The problem I have is applying (backquote) to the lists once I've read them in.

Here is the code I have been using to experiment with solutions:

(defvar x 23)

(defun read-from-file (file)
  (with-temp-buffer
    (insert-file-contents file)
    (read (current-buffer))))

;;(defun apply-macro (macro arg-list)
;;  (eval
;;   `(,macro ,@(loop for arg in arg-list
;;                    collect `(quote ,arg)))))

(defvar parts (mapcar 'read-from-file (directory-files "./parts/" "parts/" "\.part$")))

;;(apply-macro 'backquote parts)

parts 

This code relies on "data" files in a subdirectory called parts/. Here are some samples:

1. parts/one.part
   `( ("one" "two" "three") ("ten" ,x "twelve") )
   NOTE: the ,x in this one. I want to have this evaluated after the expression is read from the file.
2. parts/two.part
   ( ("four" "five" "six") (2 4 6) (9 87 6) )
3. parts/three.part
   (("seven" "eight" "nine"))

Reading the "part" files is no problem. The (defvar parts (mapcar ... ) expression works.

The problem is that once I have the lists in the parts var, I cannot find a way to get the ,x evaluated as it would be if the whole list was backquoted and not read from files.

I have tried a solution suggested in this question. You can see the apply-macro function commented out in my code above. When I run it I get:

Debugger entered--Lisp error: (wrong-number-of-arguments #[(structure) "\301!A\207" [structure backquote-process] 2 1628852] 3)
  #[(structure) "\301!A\207" [structure backquote-process] 2 1628852]((quote (\` (("one" "two" "three") ("ten" (\, x) "twelve")))) (quote (("seven" "eight" "nine"))) (quote (("four" "five" "six") (2 4 6) (9 87 6))))
  (backquote (quote (\` (("one" "two" "three") ("ten" (\, x) "twelve")))) (quote (("seven" "eight" "nine"))) (quote (("four" "five" "six") (2 4 6) (9 87 6))))
  eval((backquote (quote (\` (("one" "two" "three") ("ten" (\, x) "twelve")))) (quote (("seven" "eight" "nine"))) (quote (("four" "five" "six") (2 4 6) (9 87 6)))))
  apply-macro(backquote ((\` (("one" "two" "three") ("ten" (\, x) "twelve"))) (("seven" "eight" "nine")) (("four" "five" "six") (2 4 6) (9 87 6))))
  eval-region(648 678 t #[257 "\300\242b\210\301\207" [(678) (apply-macro (quote backquote) parts)] 2 "\n\n(fn IGNORE)"])  ; Reading at buffer position 651
  eval-defun-2()
  #[257 "\211\203

Answer 1

Backquote does interesting things. In Emacs lisp the returned value from reading a quasi-quoted list is a list of the following structure:

ELISP> (defvar x (car (read-from-string "`(1 2 ,x)")))

ELISP> (car x)
\`

ELISP> (cdr x)
((1 2
    (\, x)))

ELISP> (caddr (cadr x))
(\, x)

ELISP> (consp (caddr (cadr x)))
t

So, if you intend on using quasi-quoted lists, you might need to perform the replacement your self. For example, you can do:

(defun replace-item (item new-item seq)
  (let ((found-item (member item seq)))
    (when found-item
      (setf (car found-item) new-item))
    seq))


ELISP> (replace-item '(\, x) 'z (cadr x))
(1 2 z)

PS. Common Lisp does something weird with comma characters, after reading the same list ,X becomes an object of type SB-IMPL::COMMA (in SBCL): it's neither a symbol, nor a pair.

PPS. Somehow those quasi-quotes and commas are treated specially by the reader-evaluator, to the point that the combo (eval (read <...>)) does not produce the same result as internal evaluator.

Something that works

While playing around with back-quotes and commas, I found that the following works, although it's quite a bit of hack.

First, don't back-quote your structures: it doesn't do any harm, but it wouldn't introduce anything either. Just have (a b ,c).

When you read it (either with read from file or with read-from-string), it will be transformed into:

ELISP> (setq x (car (read-from-string "(a b ,c)")))
(a b
   (\, c))

Now, the piece of magic: there is macro backquote that does the substitution, but it accepts a structure: it does not evaluate its argument, so to make it act on x have to do the following:

ELISP> (let ((c 10)) (eval `(backquote ,x)))
(a b 10)

As you can see (\, c) was replaced by local binding of c as wanted.

PPPS. One would expect that reading from string "(a b ,c)"would produce(backquote (a b ,c))` but it doesn't.

I hope this provides the answer.