Clojure Core Cache Stackoverflow

Question

Here's a namesake question for this site.

Why does this piece of code throw a StackOverflow exception in Clojure?

(require [clojure.core.cache :as cache])

(def C (atom (cache/fifo-cache-factory {} :threshold 1E7)))

(doseq [i (range 1 1E6)]
  (swap! C cache/miss i i))

And I get the following. I didn't do any concat operations in my code. This is very reproducible. I'm using clojure 1.7 on Java 8.

Caused by: java.lang.StackOverflowError
    at clojure.lang.RT.seq(RT.java:507)
    at clojure.core$seq__4128.invoke(core.clj:137)
    at clojure.core$concat$fn__4215.invoke(core.clj:691)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.RT.seq(RT.java:507)
    at clojure.core$seq__4128.invoke(core.clj:137)
    at clojure.core$concat$fn__4215.invoke(core.clj:691)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.RT.seq(RT.java:507)

...........
...........
    at clojure.core$concat$fn__4215.invoke(core.clj:691)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.RT.seq(RT.java:507)
    at clojure.core$seq__4128.invoke(core.clj:137)
    at clojure.core$concat$fn__4215.invoke(core.clj:691)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.RT.seq(RT.java:507)
    at clojure.core$seq__4128.invoke(core.clj:137)
    at clojure.core$concat$fn__4215.invoke(core.clj:691)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.RT.seq(RT.java:507)
    at clojure.core$seq__4128.invoke(core.clj:137)
    at clojure.core$concat$fn__4215.invoke(core.clj:691)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)
    at clojure.lang.LazySeq.seq(LazySeq.java:49)
    at clojure.lang.RT.seq(RT.java:507)
    at clojure.core$seq__4128.invoke(core.clj:137)
    at clojure.core$concat$fn__4215.invoke(core.clj:691)
    at clojure.lang.LazySeq.sval(LazySeq.java:40)

score 4 · Accepted Answer · edited May 23 '17 at 11:59

4

core.cache is making a classic lazy-sequence mistake: concatenating many lazy-sequences together without ever looking at the results. FIFOCache stores a queue of past misses, and when it runs out of space in the cache, it evicts the oldest item in that queue. However, instead of using a real queue structure for this, it uses a lazy sequence and concats to the end, with results similar to those in, for example, Recursive function causing a stack overflow.

I would say this is a bug in core.cache; you can file an issue against its JIRA, or work around it by not using such large FIFO caches: it probably works up to sizes of a couple thousand or so.

edited May 23 '17 at 11:59

Community

1
1

answered Aug 07 '16 at 05:57

amalloy

89,153
8
140
205

Alright.. for anybody who is interested in a fix for this, it is here: https://github.com/ravigit/core.cache . The changes are here: https://github.com/ravigit/core.cache/commit/c8a60dda1d3ff74b7f91eacf97ebcaa1eadb2c30?diff=split . Use them at your own risk! – Ravi Aug 08 '16 at 21:35
1

You also need to fix `prune-queue`, since it converts the queue into a lazy sequence by calling `remove`. – amalloy Aug 09 '16 at 01:06

Clojure Core Cache Stackoverflow

1 Answers1