47

Basically I'm wondering why this doesn't output anything:

tail --follow=name file.txt | grep something | grep something_else

You can assume that it should produce output I have run another line to confirm

cat file.txt | grep something | grep something_else

It seems like you can't pipe the output of tail more than once!? Anyone know what the deal is and is there a solution?

EDIT: To answer the questions so far, the file definitely has contents that should be displayed by the grep. As evidence if the grep is done like so:

tail --follow=name file.txt | grep something

Output shows up correctly, but if this is used instead:

tail --follow=name file.txt | grep something | grep something

No output is shown.

If at all helpful I am running ubuntu 10.04

kenorb
  • 155,785
  • 88
  • 678
  • 743
radman
  • 17,675
  • 11
  • 42
  • 58

5 Answers5

99

You might also run into a problem with grep buffering when inside a pipe. ie, you don't see the output from 

   tail --follow=name file.txt | grep something > output.txt

since grep will buffer its own output.

Use the --line-buffered switch for grep to work around this:

tail --follow=name file.txt | grep --line-buffered something > output.txt

This is useful if you want to get the results of the follow into the output.txt file as rapidly as possible.

simonc
  • 1,006
  • 7
  • 2
  • Just checked back on this and checked out your solution. Much cleaner and precise than my workaround, marked as correct solution. – radman Aug 29 '12 at 00:19
7

Figured out what was going on here. It turns out that the command is working it's just that the output takes a long time to reach the console (approx 120 seconds in my case). This is because the buffer on the standard out is not written each line but rather each block. So instead of getting every line from the file as it was being written I would get a giant block every 2 minutes or so.

It should be noted that this works correctly:

tail file.txt | grep something | grep something

It is the following of the file with --follow=name that is problematic.

For my purposes I found a way around it, what I was intending to do was capture the output of the first grep to a file, so the command would be:

tail --follow=name file.txt | grep something > output.txt

A way around this is to use the script command like so:

script -c 'tail --follow=name file.txt | grep something' output.txt

Script captures the output of the command and writes it to file, thus avoiding the second pipe.

This has effectively worked around the issue for me, and I have explained why the command wasn't working as I expected, problem solved.

FYI, These other stackoverflow questions are related:
Trick an application into thinking its stdin is interactive, not a pipe
Force another program's standard output to be unbuffered using Python

Community
  • 1
  • 1
radman
  • 17,675
  • 11
  • 42
  • 58
-1

grep pattern filename | grep pattern | grep pattern | grep pattern ......

Sanjay
  • 7
  • 1
-1

You do know that tail starts by default with the last ten lines of the file? My guess is everything the cat version found is well into the past. Try tail -n+1 --follow=name file.txt to start from the beginning of the file.

geekosaur
  • 59,309
  • 11
  • 123
  • 114
-1

works for me on Mac without --follow=name

bash-3.2$ tail delme.txt | grep po
position.bin
position.lrn
bash-3.2$ tail delme.txt | grep po | grep lr
position.lrn
Billy Moon
  • 57,113
  • 24
  • 136
  • 237
  • Tried this and it actually works for me too, however the --follow=name part is vital to what I want to achieve – radman Mar 25 '11 at 02:54
  • Of course it works without the follow part, because this is caused by the buffering, which is flushed once the grep (without follow) exits. – Martin C. Oct 19 '11 at 08:20