How to get the PID of a process that is piped to another process in Bash?

Question

I am trying to implement a simple log server in Bash. It should take a file as a parameter and serve it on a port with netcat.

( tail -f $1 & ) | nc -l -p 9977

But the problem is that when the netcat terminates, tail is left behind running. (Clarification: If I don't fork the tail process it will continue to run forever even the netcat terminates.)

If I somehow know the PID of the tail then I could kill it afterwards.
Obviously, using $! will return the PID of netcat.

How can I get the PID of the tail process?

Answer 1

Another option: use a redirect to subshell. This changes the order in which background processes are started, so $! gives PID of the tail process.

tail -f $1 > >(nc -l -p 9977) &
wait $!

Answer 2

Write tail's PID to file descriptor 3, and then capture it from there.

( tail -f $1 & echo $! >&3 ) 3>pid | nc -l -p 9977
kill $(<pid)

Answer 3

how about this:

jobs -x echo %1

%1 is for first job in chain, %2 for second, etc. jobs -x replaces job specifier with PID.

Answer 4

This works for me (SLES Linux):

tail -F xxxx | tee -a yyyy &
export TAIL_PID=`jobs -p`
# export TEE_PID="$!"

The ps|grep|kill trick mentioned in this thread would not work if a user can run the script for two "instances" on the same machine.

jobs -x echo %1 did not work for me (man page not having the -x flag) but gave me the idea to try jobs -p.

Answer 5

Maybe you could use a fifo, so that you can capture the pid of the first process, e.g.:

FIFO=my_fifo

rm -f $FIFO
mkfifo $FIFO

tail -f $1 > $FIFO &
TAIL_PID=$!

cat $FIFO | nc -l -p 9977

kill $TAIL_PID

rm -f $FIFO

Answer 6

Finally, I have managed to find the tail process using ps. Thanks to the idea from ennuikiller.

I have used the ps to grep tail from the args and kill it. It is kind of a hack but it worked. :)

If you can find a better way please share.

Here is the complete script:
(Latest version can be found here: http://docs.karamatli.com/dotfiles/bin/logserver)

if [ -z "$1" ]; then
    echo Usage: $0 LOGFILE [PORT]
    exit -1
fi
if [ -n "$2" ]; then
    PORT=$2
else
    PORT=9977
fi

TAIL_CMD="tail -f $1"

function kill_tail {
    # find and kill the tail process that is detached from the current process
    TAIL_PID=$(/bin/ps -eo pid,args | grep "$TAIL_CMD" | grep -v grep | awk '{ print $1 }')
    kill $TAIL_PID
}
trap "kill_tail; exit 0" SIGINT SIGTERM

while true; do
    ( $TAIL_CMD & ) | nc -l -p $PORT -vvv
    kill_tail
done

Answer 7

ncat automatically terminates tail -f on exit (on Mac OS X 10.6.7)!

# simple log server in Bash using ncat
# cf. http://nmap.org/ncat/
touch file.log
ncat -l 9977 -c "tail -f file.log" </dev/null   # terminal window 1
ncat localhost 9977 </dev/null                  # terminal window 2
echo hello > file.log                           # terminal window 3

Answer 8

Not an ideal answer, but I found a workaround for a logger daemon I worked on:

#!/bin/sh
tail -f /etc/service/rt4/log/main/current --pid=$$ | grep error

from $info tail:

--pid=PID
          with -f, terminate after process ID, PID dies

Answer 9

One way would be to simply do a ps -ef and grep for tail with your script ppid

Answer 10

Have you tried:

nc -l -p 9977 -c "tail -f $1"

(untested)

Or -e with a scriptfile if your nc doesn't have -c. You may have to have an nc that was compiled with the GAPING_SECURITY_HOLE option. Yes, you should infer appropriate caveats from that option name.

Answer 11

You could use the coproc command twice.

The given example translates to:

coproc TAIL { tail -f $1; }; exec {TAIL[1]}<&-
coproc NC { nc -v -l -p 9977; } <&"${TAIL[0]}" >&1
wait $NC_PID; echo "nc exit code: $!"
kill $TAIL_PID; echo "done"

(I've thrown a -v and a couple echo in there for troubleshooting.)

Using coproc feels a lot like using Popen() in various other scripting languages.

Answer 12

bobbogo answer works but requires a intermediary pid file.

You can leverage the process substitution feature >() which works like| but without waiting for tail to finish

tail -f $1 > >(nc -l -p 9977) & pid=!

Answer 13

You may store the pid of the tail command in a variable using Bash I/O redirections only (see How to get the PID of a process in a pipeline).

# terminal window 1
# using nc on Mac OS X (FreeBSD nc)
: > /tmp/foo
PID=$( { { tail -f /tmp/foo 0<&4 & echo $! >&3 ; } 4<&0 | { nc -l 9977 ;} & } 3>&1 | head -1 )
kill $PID

# terminal window 2
nc localhost 9977

# terminal window 3
echo line > /tmp/foo

Answer 14

The --pid option to tail is your best friend here. It will allow you total control of the pipeline running in background. read the tail command options for more resilience in case your file is actively rotated by another process which might leave you tailing a inactive inode. The example below, though not used to process the data demonstrate the "imposed" restriction on the tail and the ability to tell it to exit at any time. This is used for measuring the service pressure on httpd .

  # Set the tail to die in 100 second even if we die unexpectedlly.
sleep 100 & ;  ctlpid=$!
tail -q -n 0 --follow=name --retry --max-unchanged-stats=1 --pid=$ctlpid -f  /var/log/httpd/access_log 2>/dev/null | wc –l > /tmp/thisSampleRate &
…. Do some other work
….  Can kill the pipe at any time by killing $ctlpid 
…. Calculate preassure if /tmp/thisSampleRate is ready