14

What I want to do - Start capturing the CHANGES to a log file using a custom ssh client code***. After some time (which not a fixed value and is event based), issue a command to stop tailing. This is the command I use to capture the latest changes to a log file - tail -f logfile.txt

I want to be able to end it with something like :q which I can issue from a script. I don't want to keyboard commands like ctrl + c.

*** Pseudo code for my custom ssh client code (written in an oop language)

include ssh-api
ssh = getSSHConnection();
cmd = 'cd to folder';
ssh.command(cmd);
cmd = 'tail -f log.txt';
ssh.command(cmd);
wait for special event to occur...
cmd = 'stop the tail now!'
out = ssh.command(cmd);
print "the changes made to log file were\n"  + out;

I have no write access to the server where the log file is located.

What I tried - http://www.linuxquestions.org/questions/red-hat-31/how-to-stop-tail-f-in-scipt-529419/

I am not able to understand the solution there (Its in post 2). Can someone please explain the solution or suggest a different way to do this?

Thank you.

Erran Morad
  • 4,563
  • 10
  • 43
  • 72
  • If there was a way to send the Ctrl+C from your ssh client, would that work for you? – psmears Feb 24 '15 at 22:54
  • @psmears - surely. that would work too. All I want to do is tell unix to tail a file, stop tailing it whenever I want, and get me the output between starting and ending the tailing. Thanks. – Erran Morad Feb 24 '15 at 23:30
  • @BoratSagdiyev When you say get you the output between the start and end of the tail do you mean in a file ? –  Feb 25 '15 at 09:08
  • Once the bounty expires, this should be closed as a duplicate. This is a FAQ and you would easily have found the answer by googling. – tripleee Feb 26 '15 at 06:07
  • Did you try to send à `Ctrl + C` by using `printf "\03"` ? Else, the right way to send `Ctrl-c` is `kill -INT $Pid` – F. Hauri - Give Up GitHub Feb 28 '15 at 18:54
  • 1
    possible duplicate of http://stackoverflow.com/questions/2041437/ending-tail-f-started-in-a-shell-script – tripleee Mar 04 '15 at 10:21

6 Answers6

23

Inside your script you can make use of $! variable.

# run tail -f in background
tail -f /var/log/sample.log > out 2>&1 &

# process id of tail command
tailpid=$!

# wait for sometime
sleep 10

# now kill the tail process
kill $tailpid

$! Expands to the process ID of the most recently executed background (asynchronous) command.

anubhava
  • 761,203
  • 64
  • 569
  • 643
  • Thanks. Actually, I am issuing "tail start" and "tail end" commands from another code which opens up a session. In other words, sleep is not a fixed value. Will your approach work here? – Erran Morad Feb 19 '15 at 07:19
  • `tail -f /var/log/sample.log > out 2>&1 &` results in bash out: Permission denied. Even sudo tail -f....does not help. – Erran Morad Feb 19 '15 at 07:21
  • I tried this manually before making a script. `tail -f /var/log/sample.log &` Just this command does an endless tail before I can enter `tailpid=$!`. Please help. – Erran Morad Feb 19 '15 at 07:26
  • Actually, I making my own ssh program using some ssh apis. The code goes like `ssh = getSSHConnection; out = ssh.run(string_command)`. Then the code does `print out`; I am not allowed write privilege on the server. – Erran Morad Feb 19 '15 at 07:30
  • Actually, I want to copy the changes being written to a log file. My ssh client code will read it line by line or bulk for some time (end of duration depends on an event in the client code)and output it to some file on client or console of client. I have no permission to write to the server. – Erran Morad Feb 19 '15 at 07:40
  • ok you can redirect `tail` command wherever you want. Like `tail -f /var/log/sample.log > /tmp/tail.out &` then use capture `$!` and kill it later. – anubhava Feb 19 '15 at 07:43
  • I cannot create files on the server. Any other approaches ? – Erran Morad Feb 19 '15 at 07:50
  • Thanks. I am new to unix/linux and bash. So, I'll need some help to understand. I was hoping to put the bash output to some place where my oop code could read it live or maybe read it after the tailing is complete. Looks like the output of tail is going to &1. How do I read &1? I want to be able to read the tail output and print in my OOP code. Thanks anubhava. – Erran Morad Feb 24 '15 at 22:48
  • 2
    `&1` is not a file but reference to file descriptor=1 which is stdout. You may need to read some basic tutorial on redirection in unix. If you want to keep it in a file then use `tail -f /var/log/sample.log > /tmp/file.out 2>&1 &` Assuming you have permissions to write into `/tmp/file.out` – anubhava Feb 24 '15 at 22:52
  • 1
    [Here is one tutorial on Unix redirection](http://sc.tamu.edu/help/general/unix/redirection.html) – anubhava Feb 24 '15 at 22:53
  • Thanks anubhava. I could put the output of tail into a file in tmp and then somehow read it from my OOP code. I wonder how putty gets "live and streaming" logs. Ideally, I'd like to do it like putty if its not too complicated. – Erran Morad Mar 01 '15 at 18:54
  • 1
    Let me rephrase that. Ideally, I want my code to get the output to console, the way putty does, that is live. Currently, I just put the tail output in tmp folder and then retrieve it from there. Hope that makes sense. Thank you. – Erran Morad Mar 02 '15 at 04:31
  • 1
    Actually you can choose to not to redirect in any file and just run command as `tail -f /var/log/sample.log &` but you wrote above that **this command does an endless tail**. You can also get the process id of `tail` from other putty session also using `pgrep tail` command. – anubhava Mar 02 '15 at 06:55
4

Some ways I use to work around tail -f, using .

Waiting for specific incomming:

I use this often:

sed -une '
    /DHCPOFFER/{
        s/^\(.*\) [^ ]\+ dhcpd: DHCPOFFER on \([0-9.]\+\) to \([0-9a-f:]\+\) .*$/\1 \3 -> \2/p;
        q;
    }' < <(
      tail -f /var/log/syslog
    )

With this command, I could wait for next dhcp client and get time of event, mac address and offered IP.

In a script:

#!/bin/bash

read rMac rIp rDate < <(
  sed -une '
    /DHCPOFFER/{
      s/^\(.*\) [^ ]\+ dhcpd: DHCPOFFER on \([0-9.]\+\) to \([0-9a-f:]\+\) .*$/\3 \2 \1/p;
      q;
    }' < <(
      tail -n0 -f /var/log/syslog
))

printf "Date:\t%s\nMac:\t%s\nIP:\t%s\n" "$rDate" $rMac $rIp

For having a script remotely stopped:

The fifo way:

mkfifo /tmp/holder
tail -f /var/log/syslog &
TailPid=$!
cat >/dev/null /tmp/holder
kill -9 $TailPid

... Then from elswhere:

echo >/tmp/holder

will terminate tail command.

The lockpid way

#!/bin/bash

[ -e /tmp/lockfile ] && exit
echo $$ >/tmp/lockfile
[ $(</tmp/lockfile) -ne $$ ] && exit
cd /var/log
tail -f syslog &
export tPid=$!
trap "kill $tPid;rm /tmp/lockfile;exit 0" 12
wait $tPid

Then, from elswhere:

kill -USR2 $(</tmp/lockfile)

Via SSH

The second method work fine through ssh:

ssh -T root@remoteMachine /bin/bash <<"eocmd"
    [ -e /tmp/lockfile ] && exit
    echo $$ >/tmp/lockfile
    [ $(</tmp/lockfile) -ne $$ ] && exit
    cd /var/log
    tail -f syslog &
    export tPid=$!
    trap "kill $tPid;rm /tmp/lockfile;exit 0" 12
    wait $tPid
eocmd

(care about double quote around inline script tag)

Then, from elswhere:

ssh root@remoteMachine '/bin/bash -c "kill -USR2 $(</tmp/lockfile)"'

(care about quote and double quote, in this order)

Nota about security consideration: This don't take care about security issues! Having this kind of lockfile located in /tmp could be a bad idea...

F. Hauri - Give Up GitHub
  • 64,122
  • 17
  • 116
  • 137
3

You can execute tail in the background while redirecting its output to another file (e.g. /tmp/mylog) and write the pid of the process somewhere in a pid file (e.g. ~/mytail.pid):

tail -f logfile > /tmp/mylog & echo $! > ~/mytail.pid

Next, when you want to stop it, just execute:

kill `cat ~/mytail.pid`

Then, you can see the content of the log that you gathered in the meantime (it is also a good idea to remove it later):

cat /tmp/mylog
rm /tmp/mylog # just don't forget it there
lp_
  • 1,158
  • 1
  • 14
  • 21
1

The post #2 that you are referring to does the following:has the right answer.

It says:

tail -f /etc/httpd/logs/access_log & # This starts tailing the file in background (see & at the end)
kill `ps | grep tail | awk '{print $1;}'` # This finds the process running above tail command and kills it

You may want to introduce sleep 100 or something depending on your need, otherwise, the above two command will result in tailing just for a fraction of time before killing it.

Gaurav
  • 227
  • 1
  • 3
  • 11
  • Can I issue these commands manually, one after the other for testing? – Erran Morad Feb 19 '15 at 07:34
  • Yes, but you will have hard time doing it - because the first command will continue to tail the file on your screen. You will have to copy paste the second line on the terminal and press ENTER. If you do it right, the tailing output on screen should stop. – Gaurav Feb 19 '15 at 07:56
  • 4
    That is not a good solution - it will try to kill *all* tail processes that the user has permission to kill... or indeed, any process that has 'tail' anywhere in its name! – psmears Feb 24 '15 at 22:53
  • 1
    It isn't. I was just trying to explain the 'solution' found by OP. Edited my answer to clarify that. – Gaurav Feb 25 '15 at 15:36
  • Only ps didn't worked for me. I had to use ps ax. So the command is now `kill \`ps ax | grep tail | awk '{print $1;}'\`` – blueDexter Feb 11 '21 at 10:42
1

According to your comment to @psmears, you might use CTRL+C

@psmears - surely. that would work too. All I want to do is tell unix to tail a file, stop tailing it whenever I want, and get me the output between starting and ending the tailing. Thanks. – Borat Sagdiyev 2 days ago

So you can simply launch your command within the ssh parameter.

 ssh username@remotehost tail -f /var/log/remotelog.txt | tee result.log

And when you're done, hit CTRL+C

In the previous command I used the tee command in order to see in my terminal the new lines and store them to the file.

If you want it to be this to be scriptable

You can do the following:

 ## store result to file
 FILE=result.log

 ## launch tail remote log, and store result to result.log
 ssh -n username@remote-host tail -f /path/to/remote.log > $FILE &

 ## store pid
 SSHPID=$!

 ## wait for ":q" command
 while [ "$cmd" != ":q" ]; do
    echo -ne "\n$ "
    read -n 2 cmd
 done

 ## kill ssh when you've enough
 kill $SSHPID

 ## read result
 echo "the changes made to log file were\n"
 cat $FILE

Note that if you want to separate start and stop scripts, you just have to store SSHPID in a file within the script

 echo $SSHPID > ~/.sshpid

and retrieve it from the second one

 SSHPID=`cat ~/.sshpid`
Adam
  • 17,838
  • 32
  • 54
1

Better than a kill is to let tail exit properly:

tail -n0 --pid=$(($BASHPID+1)) -F logfile | sed '/special event string in the log/q'

When piping, PIDs are sequential, so the pid of the tail will be $BASHPID and the pid of the sed will be $BASHPID+1. The --pid switch will cause tail to exit (properly!) when the sed command quits. Obviously this is a bashism.