Modifying expressions at runtime

Question

I have been banging my head at this for about 12 hours now. All I want is to evaluate an unevaluated/quoted expression that uses variables from the local scope. I know this has to be done at runtime, not in a macro. I've tried to use macros to clean it up, though.

user=> (defn replace-0 [x] (if (= 0 x) 1 x))
user=> (clojure.walk/postwalk
            replace-0 '(+ 3 (* 4 0)))
(+ 3 (* 4 1))
;;Great! The expression is modified! A macro can clean up the code:
user=> (defmacro replacer [expr]
           `(clojure.walk/postwalk replace-0 '~expr))
user=> (replacer (+ 3 (* 4 0)))
(+ 3 (* 4 1))
;;But I really want to evaluate the expression, not just create it:
user=> (defmacro replacer2 [expr]
           `(eval (clojure.walk/postwalk replace-0 '~expr)))
user=> (replacer2 (+ 3 (* 4 0)))
7
user=> (replacer2 (- 10 (* (+ 0 3) (- 2 0))))
6
;; SUCCESS!!! ....
;; Except not if the expression contains values known only at run-time.
;; This is despite the fact that the expressions are being modified
;; at run-time based on values known at run-time.

user=> (let [a 3] (replacer2 (- 10 (* a 0))))
CompilerException java.lang.RuntimeException: Unable to resolve
symbol: a in this context, compiling:(NO_SOURCE_PATH:13:1)

Eval doesn't see the local binding of a. I have tried a thousand ways. I have encountered errors for trying to embed objects in code, for not being able to rebind non-dynamic variables, and for trying to use non-global variables. I tried using declare to create dynamic variables with dynamically-generated names, and I couldn't get that to work -- the declarations worked, but the dynamic tag would be ignored (I may post a question on that one of these days).

There are actually quite a few questions on SO that run up against this problem, and in every single instance I found, solutions were presented that worked around the issue, because usually there's an easier way. But work-arounds are entirely dependent on the individual problems. And Clojure is a Lisp, a homoiconic language -- my programs should be able dynamically to modify themselves. There has got to be a way to do this, right?

Another example, this time starting with locally-bound symbols:

user=> (defn replace-map [smap expr]
          (clojure.walk/postwalk-replace smap expr))
user=> (replace-map '{s (+ 1 s)} '(+ 3 s))
(+ 3 (+ 1 s))
;; So far, so good.
user=> (defn yes-but-increment-even
         [val sym] (if (even? val) sym (list '+ 1 sym)))
user=> (defmacro foo [& xs]
         `(zipmap '~xs
                  (map yes-but-increment-even
                           (list ~@xs)
                           '~xs))))
user=> (let [a 3 b 4 c 1] (foo a b c))
{a (+ 1 a), c (+ 1 c), b b}
user=> (defmacro replacer [vs body]
         `(let [~'bod (replace-map (foo ~@vs) '~body)]
             ~'bod))
user=> (let [a 1 b 2 c 8] (replacer [a b c] (+ a b c)))
(+ (+ 1 a) b c)
;; It's working! The expression is being modified based on local vars.
;; I can do things with the expression then...
user=> (let [a 0 b 5 c 3] (str (replacer [a b c] (+ a b c))))
"(+ a (+ 1 b) (+ 1 c))"

So close, and yet so far away...

For my immediate application, I am working with ARefs:

user=> (defn foo [val sym] (if (instance? clojure.lang.ARef val)
                              (list 'deref sym)
                              sym))
user=> (let [a 1 b 2 c 8] (replacer [a b c] (+ a b c)))
(+ a b c)
user=> (let [a 0 b (ref 5) c 3] (str (replacer [a b c] (+ a b c))))
"(+ a (deref b) c)"
user=> (let [a 0 b (ref 5) c 3] (eval (bar [a b c] (+ a b c))))
CompilerException java.lang.RuntimeException: Unable to resolve
symbol: a in this context, compiling:(NO_SOURCE_PATH:146:1)

Answer 1

Your first example doesn't get at what you really want judging from your last examples. In the first example, the replacement values are known at compile time (i.e. expressions of literals), so there is an easier way:

(defmacro replacer [smap expr]
  (clojure.walk/postwalk-replace smap expr))

(let [a 3] (replacer {0 1} (- 10 (* a 0))))
;=> 7

This works because the replacement map is known at compile (macro-expansion) time.

If replacement depends on values at runtime, you'll need eval.

The Problem

This fails:

(let [a 3]
  (eval 
    (clojure.walk/postwalk-replace 
      {0 1}
      '(- 10 (* a 0)))))

The reason is that eval does not see context; that is, it can't see the binding of a.

The Trick

You can work around this by wrapping your expression in a function that takes a as an argument, do eval, then pass the value of a to the (outer) function produced by eval.

(let [a 3
      f (eval 
          (clojure.walk/postwalk-replace 
            {0 1}
            '(fn [a] (- 10 (* a 0)))))]
  (f a))
;=> 7

This is definitely working at run-time:

(def my-map {0 1})

(defn foo [] 
  (let [a 3] 
       [f (eval 
            (clojure.walk/postwalk-replace 
              my-map
             '(fn [a] (- 10 (* a 0)))))]
    (f a)))

(foo) ;=> 7

(def my-map {0 2})
(foo) ;=> 4 (without having to redefine foo)

More Complex Example

I believe for your last example you are for something like this:

(defn maybe-deref-expr 
  [vals params body] 
  (let [smap (zipmap params 
                     (map (fn [val sym] 
                            (if (instance? clojure.lang.IDeref val) 
                              (list 'deref sym) 
                              sym)) 
                          vals 
                          params))
        body* (clojure.walk/postwalk-replace smap body)
        gen (eval (list 'fn params body*))] 
    (apply gen vals)))

(def r1 (ref 1))

(def instance (maybe-deref-expr [r1 10] '[a b] '(fn [x] (+ a b x))))

(instance 100)
;=> 111