Clojure doesn't release memory

Question

I'm learning Clojure and I'm wondering why my application is consuming so much memory and not releasing it once it's done using those data structures. I imagine that there is something in Clojure that I don't know about that is making the application hold onto some stuff by mistake.

It's a simple thumbnailing app. It takes images from a directory, resizes them and puts them into another directory. It uses the javax.imageio and java.awt.image libraries.

(def thumb "path to thumbnails directory")

(defn get-thumb-filename [thumb-dir filename width]
  (str thumb-dir (fs/name filename) "_" width ".jpg"))

(defn get-thumb-filename-100 [filename]
  (get-thumb-filename thumb filename 100))

(defn small-thumbnail [filename]
  (make-thumbnail filename (get-thumb-filename-100 filename) 100))

(defn make-thumbnail [filename new-filename width]
  (let [img (javax.imageio.ImageIO/read (as-file filename))
        imgtype (java.awt.image.BufferedImage/TYPE_INT_ARGB)
        width (min (.getWidth img) width)
        height (* (/ width (.getWidth img)) (.getHeight img))
        simg (java.awt.image.BufferedImage. width height imgtype)
        g (.createGraphics simg)]
    (.drawImage g img 0 0 width height nil)
    (.dispose g)
    (javax.imageio.ImageIO/write simg "png" (as-file new-filename))))


;; `images` is a list of original filenames
;; Kick it off with...
(pmap small-thumbnail images)

This code is part of a Noir view, meaning that the application doesn't exit after the processing of all images is finished.

When I run this code, my computer starts going crazy. The fan gets loud and the main java process uses over 1GB of memory. When all images are processed, the CPU usage goes down to < 5% but the memory usage stays the same.

How can I make it release all of this memory that it clearly doesn't need for anything?

Thanks!

PS: Please let me know if you need me to fill in any details.

how are you measuring memory use? does java free memory back to the os? that would surprise me (but it might). — andrew cooke, Aug 05 '12 at 00:55
http://stackoverflow.com/questions/324499/java-still-uses-system-memory-after-deallocation-of-objects-and-garbage-collecti — andrew cooke, Aug 05 '12 at 00:58
I'm measuring memory consumption by looking at the process in Activity Monitor. It just sits there, using 1GB of RAM. — Honza Pokorny, Aug 05 '12 at 11:10
then see the link i posted above. it could be that clojure has released all the memory, but that the jvm is keeping it to re-use later. — andrew cooke, Aug 05 '12 at 14:04

score 3 · Accepted Answer · answered Aug 05 '12 at 14:44

The JVM manages it's own memory. Once it acquires memory from the operating system, it usually doesn't give it back. (Or at least I've never seen Sun's JVM do so, there might be some way to force it to...)

Anyhow, a simple way to see whether or not your program is actually leaking memory using tools that you've probably already got installed is to attach a JConsole to your JVM and just monitor the heap usage. Looking at Activity Monitor won't tell you anything useful.

http://java.sun.com/developer/technicalArticles/J2SE/jconsole.html

if you look at the link i posted in the question comments it appears that since jdk5 it is possible for it to return memory. — andrew cooke, Aug 05 '12 at 14:53

score 2 · Answer 2 · answered Aug 04 '12 at 23:41

2

If you're doing this at the repl, the results of the pmap are being stored in *1 for your later use; that won't happen in production, and you could stop it from happening at the repl with, for example, (dorun (pmap ...)).

answered Aug 04 '12 at 23:41

amalloy

89,153
8
140
205

No, this isn't at the REPL. It's inside a Noir view. Changing `(println (pmap ...))` to `(dorun (pmap ..))` seems to have no effect. How would production be different? Wouldn't I have to add a `:production` profile for leiningen? – Honza Pokorny Aug 04 '12 at 23:55

Clojure doesn't release memory

2 Answers2