3

I'm interested in following several remote files at the same time and simultanteously aggregating stats over the file. So far, I'm doing this as follows:

mkfifo mfifo
ssh -ft host1 'tail -f /path/to/log | grep something' > mfifo &
ssh -ft host2 'tail -f /path/to/log | grep something' > mfifo &
ssh -ft host3 'tail -f /path/to/log | grep something' > mfifo &
cat mfifo | awk '{x += $4; print $3} END {printf "total: %d", x}'

This pretty much works as expected, with an aggregation of the grepped logs streamed through awk. However, I'm not sure how to get the final total to be printed. I gather that I need to close the writers of the fifo, but I'm not sure how to do this. Any suggestions regarding how to do this without storing the whole stream as a file?

jonderry
  • 23,013
  • 32
  • 104
  • 171
  • store the PIDs of the backgrounded processes, then kill them when you're ready? Good luck. – shellter Jun 28 '12 at 03:04
  • The `tail -f` processes on each remote host loop forever and are attached to the fifo, so they must be killed. This should happen if each `ssh` is killed. The `ssh` process IDs can be found with (on Mac OS X) something like `killall -t yourtty ssh` (run `tty` to find the device) or by parsing `ps | grep ssh` on most systems. – Kevin Grant Jun 28 '12 at 03:07
  • Why not just omit `-f` from the invocation of tail? How do you know when you want the writers to be closed? – William Pursell Jun 28 '12 at 15:48

1 Answers1

2

Killing FIFO Writers

You can use fuser to kill the processes writing to a file. For example:

fuser -TERM -k -w mfifo; sleep 5; fuser -k -w mfifo

Note that fuser defaults to sending SIGKILL, so the example given sends an explicit SIGTERM, then waits five seconds before forcefully terminating the process. This should allow your processes to clean up after themselves, but feel free to adjust the invocation to suit.

Also, note that we're passing the -w flag, so that fuser only kills processes with write access. Without this flag, you'll also be killing cat and awk.

Todd A. Jacobs
  • 81,402
  • 15
  • 141
  • 199
  • I didn't see a -w option, but it still worked for me without the -w option. – jonderry Jun 28 '12 at 17:00
  • I found sending [`-PIPE`](http://stackoverflow.com/a/14152313/1888983) instead did what I wanted without triggering extra output. – jozxyqk Feb 03 '17 at 19:53