9

What I'm trying to do: I'm trying to understand/build a go pipeline with three stages. Stage 1 writes to channel A. Stage 2 has multiple go routines. Each go routine reads from Channel A, performs some operation and writes result to Channel Bn(Channel B1, B2..Bn). Stage 3 has creates n(=total number of Channels in Stage 2) go routines, each go routine reads from one Channel from stage 2. It is based on bounded parallelism described in https://blog.golang.org/pipelines

Issue: The pipeline works fine as expected, the operation is distributed among go routines. But when I did an escape analysis and found that channel parameter sent from one stage to another is reported as "leaking param".

Code snippet: For simplicity, I'm posting code that shows stage 1 channel creation, and a stage 2 that prints value read from stage channel 1.

package main

import "fmt"

func createStageOne(numOfJobs int) <-chan int {
   stageOneChannel := make(chan int)
   go func(nJobs int) {
      for i := 0; i < nJobs; i++ {
        stageOneChannel <- i
      }
      close(stageOneChannel)
   }(numOfJobs)
   return stageOneChannel
} // stageOneChannel closes and go routine exits once nJobs are completed 

func createStageTwo(in <-chan int, completionFlag chan struct{}) {
    go func() {
        for n := range in {
            fmt.Println("Received from stage 1 channel ", n)
        }
        completionFlag <- struct{}{}
    }()
}// Comes out of for loop when stage 1 channel closes and go routine also exits


func main() {
    numOfJobs := 10
    stageOneChannel := createStageOne(numOfJobs)

    done := make(chan struct{})
    createStageTwo(stageOneChannel, done)

    <-done
}

Here is the escape analysis result

$ go build -gcflags "-m -l"
# concurrentHTTP/stackoverflow
./pipeline.go:7:5: func literal escapes to heap
./pipeline.go:7:5: func literal escapes to heap
./pipeline.go:6:25: make(chan int) escapes to heap
./pipeline.go:17:5: func literal escapes to heap
./pipeline.go:17:5: func literal escapes to heap
./pipeline.go:16:21: leaking param: in
./pipeline.go:16:36: leaking param: completionFlag
./pipeline.go:19:16: "Received from stage 1 channel " escapes to heap
./pipeline.go:19:16: n escapes to heap
./pipeline.go:19:15: createStageTwo.func1 ... argument does not escape
./pipeline.go:29:14: make(chan struct {}) escapes to heap

Why is escape analysis reporting leaking param on in and completionFlag flag?

Jonathan Hall
  • 75,165
  • 16
  • 143
  • 189
object
  • 796
  • 1
  • 12
  • 21
  • Yes. What is the question? – Volker Feb 11 '19 at 03:19
  • 1
    @Volker Does leaking param in escape analysis mean resource leak? Go routines exit as expected, and channels won't block forever here. Then why is it shown as leak? Am I missing something? – object Feb 11 '19 at 03:32
  • "Does leaking param in escape analysis mean resource leak?" No, of course not. Why do you think that? – Volker Feb 11 '19 at 04:56
  • 4
    "Why would you think that?" It's reasonable to assume that `leaking` is bad and related to its stem `leak`. I am struggling to think of an example context where leaking is a good thing, e.g leaking bucket, leaking gas tank, _taking a leak_, leaking capacitor, leaky boat, leaky abstraction. It may be obvious to high performance go experts, but for the rest of us it would be helpful to link to docs and provide brief clarification of what `leaking param` refers to. – Davos Oct 31 '19 at 12:46

1 Answers1

14

The parameters in question are channels which are reference types. They are captured within the closure created in createStageTwo() and can continue to be used in the go routine for that closure when createStageTwo() returns. Hence they are signaled as leaking parameters. If they were not then they would be placed on the stack and become invalid when main() finished with them.

This does not mean you have a problem. Escape analysis is not there to detect resource leaks and most people would never need to use it. (It can be useful for performance problems where something is placed on the GC heap which you don't expect.)

(Sorry @Volker I originally posted my answer in the comments which left your question hanging.)

Andrew W. Phillips
  • 3,254
  • 1
  • 21
  • 24