I have a shell script that is doing some heavy work and keeping track of what it's doing using counters. The heavy work needs to be done for around 30 cases, and as the machine I'm running it on has 48 cores, it is easy to parallelise: I simply run the function in the background, creating a subshell. However, this means I cannot increment a global variable:
$ cat mwe.sh
#!/bin/bash
counter1=0
counter2=0
counter3=0
func() {
# do some heavy work which may increase one counter
counter2=$((counter2 + 1))
}
func &
func &
func &
wait
echo $counter1 $counter2 $counter3
$ ./mwe.sh
0 0 0
Bash functions do not have return values as such. Answers to Counter increment in Bash loop not working suggest to write to a file and then read from there, which would require a temporary file for each individual call, which involves some overhead. Other suggestions are to use echo to "return" a string, but if I use stdout for that, the functions cannot write anything else to stdout.
How can I keep track of global counters in a bash shell script, when the heavy work is done in subshells operating in parallel? Is there some way to open a dedicated stream/pipe for each function call, to which the function can write and from which the caller can read? Or is there some other way in which I can keep track of this, without writing a file for each call?
