Interpreting pprof heap diagrams

Question

When I profile the heap in go with pprof I get the following:

However, I'm not clear on how to interpret that visualization. In particular:

"The memory next to the arrows means _____ and the memory inside of a box means ______. So when a box has multiple arrows from it, it means _____, and when it has multiple arrows to it, it means _____".

Oh, and the context is that one instance of this binary is slowly leaking memory. I have this snapshot, and am going to grab another one in several hours compare them. Unless it turns out the leak isn't in the heap, then I'm not sure what to do next. — Kyle Brandt, Mar 08 '16 at 15:28
Since Go has a precise GC, a "leak" almost always involves goroutines that aren't returning. A stack trace is often more helpful with this than a heap profile. — JimB, Mar 08 '16 at 16:34
@JimB I monitor the number of goroutines though (ironically, using the thing that is slowly consuming more memory), and the number of routines is stable. — Kyle Brandt, Mar 08 '16 at 16:43

score 17 · Accepted Answer · edited Dec 19 '16 at 13:18

the perceived cumulative memory consumption of this function including all of its subfunctions
the perceived memory consumption of this function excluding all of its subfunctions
that it calls multiple subfunctions
that multiple functions call it

No difference between the meaning of the arrows between the heap and the CPU profiling mode.

You may be interested in the output of the top pprof command, which can give you results in the following form:

9701.61kB of 9701.61kB total (  100%) Dropped 112 nodes (cum <=
48.51kB) Showing top 10 nodes out of 29 (cum >= 3611.54kB)
      flat  flat%   sum%        cum   cum%
 4549.72kB 46.90% 46.90%  4549.72kB 46.90%  mystery.function
 2587.52kB 26.67% 73.57%  2587.52kB 26.67%  reflect.unsafe_NewArray
 1024.02kB 10.56% 84.12%  1024.02kB 10.56%  encoding/xml.copyValue
     514kB  5.30% 89.42%      514kB  5.30%  compress/gzip.NewReader
     514kB  5.30% 94.72%      514kB  5.30%  net/http.(*Transport).dialConn
  512.34kB  5.28%   100%   512.34kB  5.28%  runtime.makeslice
         0     0%   100%      514kB  5.30%  bytes.(*Buffer).ReadFrom
         0     0%   100%  3611.54kB 37.23%  encoding/xml.(*Decoder).Decode
         0     0%   100%  3611.54kB 37.23%  encoding/xml.(*Decoder).DecodeElement
         0     0%   100%  3611.54kB 37.23%  encoding/xml.(*Decoder).unmarshal

And here are (IMHO) the two best references for Go heap profiling with pprof:

Interpreting pprof heap diagrams

1 Answers1