a version of `sequence` that doesn't do chunking

Question

I'd like to have a version of sequence that doesn't do the chunking of 32 elements. Currently, this code will output

(def peek #(doto % (print " ")))
(def pause #(do (Thread/sleep 10)
                %))

(take 2 (->> (range 100)
             (sequence (comp (map peek)
                             (map pause)
                             (map inc)))))

;; prints 0 1 2 3 4 <..etc..> 32
;; => (0, 1)

I'd like a version of it so that it only iterates through the elements that it needs

(take 2 (->> (range 100)
             (iter-sequence  (comp (map peek)
                             (map pause)
                             (map inc)))))

;; prints 0 1
;; => (0, 1)

Is there a way to do this?

I'd guessing it has something to do `iterate` but I'd like to be able to use transducers on it as well. — zcaudate, Aug 19 '20 at 07:30
See https://stackoverflow.com/questions/12412038/in-clojure-are-lazy-seqs-always-chunked – Chousers response in particular. — Stefan Kamphausen, Aug 19 '20 at 07:59
@stefan. this doesn't work because it still generates a seq. I'd like it to generate an iterator (as sequence does) but a non chunking version. — zcaudate, Aug 19 '20 at 08:38
@cfrick if you replace `take 2` with `first`, it prints 32 elements so the culprit is definitely sequence. — zcaudate, Aug 19 '20 at 09:10
@zcaudate Yes you are right - my testing was flawed. `sequence` code actually has the chunkedIterator in plain sight — cfrick, Aug 19 '20 at 09:15

score 3 · Answer 1 · answered Aug 19 '20 at 09:40

I had to change a couple of things to get it working. The first is to cut and paste sequence code and replace clojure.lang.RT/chunkIteratorSeq with an alternative version of clojure.lang.IteratorSeq that has exposed public constructor methods.

The reason being is that the clojure.lang.IteratorSeq/create has a check to iter.next() on L27 which will block if the source is blocking.

(defn seqiter
  {:added "1.0"
   :static true}
  ([coll] coll)
  ([xform coll]
   (IteratorSeq.
    (TransformerIterator/create xform (clojure.lang.RT/iter coll))))
  ([xform coll & colls]
   (IteratorSeq.
    (TransformerIterator/createMulti
     xform
     (map #(clojure.lang.RT/iter %) (cons coll colls))))))

the source is here: https://gist.github.com/zcaudate/269d8750b6316166f14be15bfb8aa500#file-iteratorseq-java-L48 — zcaudate, Aug 19 '20 at 12:04
also, https://github.com/cgrand/xforms/blob/master/src/net/cgrand/xforms.cljc#L555 does the same thing. — zcaudate, Aug 20 '20 at 04:29

a version of `sequence` that doesn't do chunking

1 Answers1