4

I'm playing with Go to understand its features and syntax. I've done a simple producer-consumer program with concurrent go functions, and a priority buffer in the middle. A single producer produces tasks with a certain priority and send them to a buffer using a channel. A set of consumers will ask for a task when idle, received it and consume it. The intermediate buffer will store a set of tasks in a priority queue buffer, so highest priority tasks are served first. The program also prints the Garbage collector activity (how many times it was invoked and how many time it took to collect the garbage).

I'm running it on a Raspberry Pi using Go 1.1.

The software seems to work fine but I found that at SO level, htop shows that there are 4 processes running, with the same memory use, and the sum of the CPU use is over 100% (the Raspberry Pi only has one core so I suppose it has something to do with threads/processes). Also the system load is about 7% of the CPU, I suppose because of a constant context switching at OS level. The GOMAXPROCS environment variable is set to 1 or unset.

Do you know why Go is using more than one OS process?

Code can be found here: http://pastebin.com/HJUq6sab

Thank you!

EDIT:

It seems that htop shows the lightweight processes of the system. Go programs run several of these lightweight processes (they are different from the goroutines threads) so using htop shows several processes, while psor top will show just one, as it should be.

siritinga
  • 4,063
  • 25
  • 38

1 Answers1

5

Please try to kill all of the suspect processes and try again running it only once. Also, do not use go run, at least for now - it blurs the number of running processes at minimum.

I suspect the other instances are simply leftovers from your previous development attempts (probably invoked through go run and not properly [indirectly] killed on SIGINT [hypothesis only]), especially because there's a 1 hour "timeout" at the end of 'main' (instead of a proper synchronization or select{}). A Go binary can spawn new threads, but it should never create new processes unless explicitly asked for that. Which is not the case of your code - it doesn't even import "os/exec" or "syscall" in the first place.

If my guess about the combination of go run and using a long timeout is really the culprit, then there's possibly some difference in the RP kernel wrt what the dev guys are using for testing.

zzzz
  • 87,403
  • 16
  • 175
  • 139
  • Thanks. I ran it with go build and then execute it, and all process disappear when I kill the program, so it is not the case. The PID of all 6 processes are consecutive: http://imageshack.us/photo/my-images/7/pc2r.png/ however, I cannot see all those processes using ps or top, maybe they are linked to the main process somehow... – siritinga May 26 '13 at 09:19
  • Mmm it seems that I was wrong. htop shows the OS light-weight threads https://en.wikipedia.org/wiki/Lightweight_process and not only the kernel threads (shown by top or ps), and Go seems to create several of these light-weight threads. I found the answer here: http://stackoverflow.com/questions/13234749/golang-how-to-verify-number-of-processors-on-which-a-go-program-is-running – siritinga May 26 '13 at 09:49
  • Is 4 the maximum number of lightweight threads pre executable on the RasPi? Have you tried with the pprof tools? – Intermernet May 26 '13 at 10:00
  • I haven't tried pprof (I don't know how to use them yet) but I've seen up to 6 lightweight threads in htop for my program, it looks like they are created on-demand. – siritinga May 26 '13 at 11:25
  • 3
    LWPs are, I suppose, not of a concern and might just confuse you. Go program creates threads on demand (eg. a syscall is, AFAIK, executed in a new thread). Then Go also creates goroutines. It's hard to guess if some tool can show a goroutine as a LWP. Anyway, the number of "live" goroutines is not very deterministic. 'main' has one, GC has at least one, every timer use one, ever 'go' statement creates one, some stdlib objects create goroutines, etc. I would stay with process == what ps (or top) shows. Do those tools show one or more processes of your program? – zzzz May 26 '13 at 11:41
  • Thanks jnml. No, top and ps shows just one. I was confused at first because I usually use htop, but it seems I was deceived by the lightweight processes. Thank you all! – siritinga May 26 '13 at 12:34