3

Apologies for duplicates but I haven't been able to solve this issue from reading other posts. I am trying to run a computationally intensive iterative algorithm. The next iterate is given by a function "update" and I collect all the iterates via

(def iterates (doall (take 1000 (iterate update initial-state))))

where initial-state is the zeroeth iterate with which I am initializing the algorithm. Unfortunately I cannot supply a minimal example... update is quite a complicated function which takes a map datastructure as input and outputs an updated map datastructure. Code runs fine for total iterations up to 800 but then produces the error below for anything larger. General opinion is that merely increasing the size available to the garbage collector is not sufficient, as this is symptomatic of a deeper problem (perhaps). In other posts the problem presented are very specific, yet my update function is very complex. How then can I go about solving this in the generality provided here? Are there certain tooling utilities I can use?

  1. Caused by java.lang.OutOfMemoryError GC overhead limit exceeded

           Double.java:  519  java.lang.Double/valueOf
      Numbers.java:  611  clojure.lang.Numbers$DoubleOps/multiply
      Numbers.java:  148  clojure.lang.Numbers/multiply
          core.clj:   21  ssm4clj.core/corr
          core.clj:    6  ssm4clj.core/corr
          core.clj:   42  ssm4clj.core/innovation
          core.clj:   40  ssm4clj.core/innovation
          core.clj: 2522  clojure.core/partial/fn
          core.clj: 2646  clojure.core/map/fn
      LazySeq.java:   40  clojure.lang.LazySeq/sval
      LazySeq.java:   49  clojure.lang.LazySeq/seq
           RT.java:  521  clojure.lang.RT/seq
          core.clj:  137  clojure.core/seq
          core.clj: 2654  clojure.core/map/fn
      LazySeq.java:   40  clojure.lang.LazySeq/sval
      LazySeq.java:   49  clojure.lang.LazySeq/seq
         Cons.java:   39  clojure.lang.Cons/next
           RT.java:  688  clojure.lang.RT/next
          core.clj:   64  clojure.core/next
     protocols.clj:  168  clojure.core.protocols/fn
     protocols.clj:  124  clojure.core.protocols/fn
     protocols.clj:   19  clojure.core.protocols/fn/G
     protocols.clj:   31  clojure.core.protocols/seq-reduce
     protocols.clj:   75  clojure.core.protocols/fn
     protocols.clj:   75  clojure.core.protocols/fn
     protocols.clj:   13  clojure.core.protocols/fn/G
          core.clj: 6545  clojure.core/reduce
          core.clj: 6527  clojure.core/reduce
          core.clj:  347  ssm4clj.core/mean-conditional
          core.clj:  319  ssm4clj.core/mean-conditional
              REPL:  332  ipp4clj.multiplexing/update-single-gp-mean
              REPL:  329  ipp4clj.multiplexing/update-single-gp-mean
Lindon
  • 1,292
  • 1
  • 10
  • 21
  • 1
    How much memory would a set of 1000 states take up? Is it possible that you simply don't have enough RAM to hold this much data and the additional garbage that is created by the calculations that you're doing? – Sam Estep Feb 26 '17 at 20:37
  • Do you need all the states or maybe just one that reaches a goal? I wonder if you could use `drop` as well as `take` to discard states as you go along?? Or `drop-while` combined with `first`? – Chris Murphy Feb 26 '17 at 21:05
  • @SamEstep, when I "top" the process, the percentage in the %MEM columns is around 30 for process in question. Checking the docs of top, %MEM displays the task's currently used share of available physical memory. It is my understanding that the process is only using 30% of my RAM. – Lindon Feb 26 '17 at 21:21
  • @ChrisMurphy Unfortunately in this application I need all the iterates. – Lindon Feb 26 '17 at 21:21
  • 1
    @Lindon Ah yes, you're right; I had forgotten that there were other limits for JVM memory. How much does 30% of your RAM amount to? You may need to increase the maximum heap size as explained [here](http://stackoverflow.com/a/3067285/5044950). But my question still stands: how much memory would a set of 1000 states take up? Since you haven't specified your state structure and `update` function, we can't really make judgments without that sort of information. – Sam Estep Feb 26 '17 at 22:39
  • @SamEstep The machine upon which I'm running this code has 16GB of RAM (obtained from executing `free -mt` in a terminal). I did not find an easy way of querying the size of an object from inside the clojure repl, so lets for the sake of argument assume that 1000 states takes up a lot of memory. If this is the case, what would you suggest I do? Manually increase the maximum heap size? – Lindon Feb 27 '17 at 00:58
  • @Lindon OK, yeah; generally speaking, the only thing I can think of would be to manually increase the maximum heap size; if you could give more information on your state structure and `update` function, or find a way to figure out a rough estimate of how much memory would be taken up by 1000 states, I or someone else could potentially give advice on how optimize, but it looks like you're probably going to need to increase your maximum heap size. – Sam Estep Feb 27 '17 at 02:26
  • Do 500 iterations, send them off (or store them) but keeping the last one, then go again for the next 500 (or maybe smaller amount if the data structures are getting bigger) using the last from last time as the first from this time. Also the data structures could be made more canonical (i.e. take up less space and be more suited to the problem). Also JVM memory of course. – Chris Murphy Feb 27 '17 at 02:29
  • 2
    Have tried to look into the running JVM process with jvisualvm? With Oracle JDK it is included, with OpenJDK it is usually a separate package. – Stefan Kamphausen Feb 27 '17 at 15:32

1 Answers1

1

To go along with @StefanKamphausen's comment. I think jvisualvm would also help you out quite a bit. Here is a walk through of usage.

Stephen Greene
  • 69
  • 3