Understanding list comprehension modifier order in Clojure

Question

I'm currently learning Clojure and am stuck with list comprehension.

;; https://stackoverflow.com/a/7625207/4110233
(defn gen-primes "Generates an infinite, lazy sequence of prime numbers"
  []
  (letfn [(reinsert [table x prime]
            (update-in table [(+ prime x)] conj prime))
          (primes-step [table d]
            (if-let [factors (get table d)]
              (recur (reduce #(reinsert %1 d %2) (dissoc table d) factors)
                     (inc d))
              (lazy-seq (cons d (primes-step (assoc table (* d d) (list d))
                                             (inc d))))))]
    (primes-step {} 2)))

(defn prime-factors-not-working [x]
  (for [y (gen-primes) 
        :when  (= (mod x y) 0) 
        :while (< y (Math/sqrt x))] 
     y))

(defn prime-factors-working [x]
  (for [y (gen-primes) 
        :while (< y (Math/sqrt x))
        :when  (= (mod x y) 0)]
     y))

(prime-factors-working 100)
;; ↪ (2 5)

(prime-factors-not-working 100)
;; Goes into infinite loop

(gen-primes) is a lazy sequence of prime numbers. The only difference between the working and not-working sequences is the order of the modifiers while and when. Why does the not-working implementation go into an infinite loop?

score 5 · Answer 1 · answered Jul 19 '22 at 13:17

The not working variant expands conceptually (but not factually) into this:

(loop [ys (gen-primes)
       result []]
  (if (seq ys)
    (let [y (first ys)]
      (if (= (mod x (first ys)) 0) ;; Can be replaced with `(zero? (mod x y))` BTW.
        (if (< y (Math/sqrt x))
          (recur (next ys) (conj result y))
          result)
        (recur (next ys) result)))
    result))

As you can see, if (mod x (first ys)) is not 0, it will go to the next number - without checking for that <, going forever.

When you exchange :when and :while, the checks in the pseudo-expansion above are also swapped - stopping the iteration once y reaches the square root of x.

score 4 · Accepted Answer · edited Jul 19 '22 at 14:47

the macro expansion of for is sensitive to the order in which you put :when and :while. macroexpanding gives slightly different code.

you can go very far in clojure without relying on complicated macros beyond defn, for is not very common, and this isn't a usecase where it is clearly advantagous over map->filter->take

good expansion: line 28:

(when (< y (Math/sqrt x)) ; XXX
  (if (= (mod x y) 0)     ; XXX
    (do
      (chunk-append b__86695 y)
      (recur (unchecked-inc i__86694)))
    (recur (unchecked-inc i__86694))))

bad expansion: line 28:

(if (= (mod x y) 0)         ; XXX
  (when (< y (Math/sqrt x)) ; XXX
    (do
      (chunk-append b__86666 y)
      (recur (unchecked-inc i__86665))))
  (recur (unchecked-inc i__86665)))

you can learn about the implementation of the for macro by going to it's source code (your editor should have a way for navigating to defs of symbols) https://github.com/clojure/clojure/blob/master/src/clj/clojure/core.clj#L4654

there is nothing about clojure that requires it's macros to write code in the way you think they should, though in this case it may be a bug in for, it's hard to tell. some use cases may want to be able to when while and when are written.

since this is about learning, and macros are pretty much magic unless you see the code they write out, i think the best way to learn is to figure out how to view the expanded macro forms in your code. this is generally how macros are debugged.

link to topic on macro expansion how is a macro expanded in clojure?

documentation for expanding macros in cider (emacs clojure editor) https://docs.cider.mx/cider/debugging/macroexpansion.html

@cfrick thanks for making the macro expands look pretty – PPPaul Jul 19 '22 at 17:16 — PPPaul, Jul 19 '22 at 17:16

Understanding list comprehension modifier order in Clojure

2 Answers2