5

The basic question comes along with the quotes from the GHCI interpreter. Yes, I know that the supposed use of code is compiling it into executable, however, as I am a beginner in Haskell, GHCI seems to be perfect to learn some Haskell.

When I use the heavy-computational load algortihms, I sometimes come across messages like the following:

Prelude Data.List> foldl' (+) 0 [1..2*10^8]
20000000100000000
it :: (Num b, Enum b) => b
(8.35 secs, 17,600,063,056 bytes)

At the same time, I only have 8 GB of RAM physically installed.

If memory usage is over a certain limit (which I'm not quite sure how many that is, but suggest that's all memory that can be used), the Stack Overflow exception is thrown, like illustrated below:

Prelude> foldl (+) 0 [1..5 * 10^7]
*** Exception: stack overflow

However, how is it possible that the program uses > 16 GBs of memory while my RAM is totalling to 8 GB and that without throwing Stack Overflow exception?

Nick M
  • 91
  • 6
  • 2
    You are reading two things that both don't mean what you might intuitively think they mean. In your first message, that does NOT necessarily mean that you had peak usage on 17.6 GB, only that there were 17.8 GB allocated. Your second message with the Stack Overflow _is_ memory related, but it's related to the deferred computations that you have. If you made that strict, you'd have no problems with the same operation. – Marc Talbot Mar 07 '19 at 12:43
  • @Marc Thank you for the explanation on the memory display matter, that does not seem to be confusing now! – Nick M Mar 07 '19 at 13:05
  • 1
    I know you mentioned it in your answer, but note that there might be a huge difference between GHCi and compiled code, especially when optimization is turned on. I'd expect your code to run using a constant amount of memory when properly compiled. If the right optimizations kick in, the `[1..2*10^8]` list cells will not even be allocated, essentially reducing your code to a sort of "for loop" as in imperative languages. – chi Mar 07 '19 at 15:22

1 Answers1

7

The magic of garbage collection: the number reported by :set +s that you show tells the total number of bytes requested from the allocator, even if they were later deallocated and returned to the allocator to be reused before the computation finished.

Daniel Wagner
  • 145,880
  • 9
  • 220
  • 380
  • Thank you for the brief and simple explanation on the matter! Other than that, is there a (built-in) way to actually measure the peak memory consumption of the process? – Nick M Mar 07 '19 at 13:04
  • 1
    @NickM There are external tools like `time` that will tell you peak memory usage of a process; but you'll need to run just one thing at a time that way. – Daniel Wagner Mar 07 '19 at 13:35
  • 1
    @NickM: For detailed analysis, you can also look into GHC’s built-in profiling tools—you can find more about it in the docs, but briefly, you compile your program with profiling enabled (e.g. the `--enable-executable-profiling` and `--enable-library-profiling` flags for Stack/Cabal/`Setup.hs`) then run it with different RTS options to produce different reports (such as `+RTS -hy` to profile by type) in `.hp` “heap profile” format, which can be converted for pretty display in a PostScript/PDF viewer using the `hp2ps` utility. – Jon Purdy Mar 07 '19 at 17:21
  • @Jon thanks a bunch for pointing out profiling can be performed using the built-in tools, will try to make use of that – Nick M Mar 07 '19 at 21:35