4

I use the following command to see which VirtualHosts are getting traffic on my server (alternatives welcome). 

 ngrep 'Host:' port 80 -t  -W byline -q | grep  '^Host'

I want to be able to analyse the number of hits in a time interval while also seeing what's going on. To do this I have been trying to write the output to a file while also displaying it using tee.

 ngrep 'Host:' port 80 -t  -W byline -q | grep  '^Host' | tee ~/hosts.log

However, nothing is printed to screen or logged to file.

I have tried different file locations, outside of my screen session - with no difference. Does grep need special handling?

PeterB
  • 2,212
  • 2
  • 21
  • 33
  • 1
    Check which `tee` you are using with `which tee`. Make sure it's actually the `tee` command and not something else. – Tripp Kinetics Aug 05 '15 at 17:50
  • possible duplicate of [Force line-buffering of stdout when piping to tee](http://stackoverflow.com/questions/11337041/force-line-buffering-of-stdout-when-piping-to-tee) – Don Branson Aug 05 '15 at 19:09

2 Answers2

4

There is line buffering at play on the part of both ngrep and grep. Here is how you can get aroundthe issue you’re seeing:

% sudo ngrep 'Host:' port 80 -t -W byline -d lo -q -l |grep --line-buffered '^Host:' |tee foo.log

Notice that ngrep is using -l to make stdout line buffered, and grep uses --line-buffered for the same.

(I added the -d lo just for my own testing.)

Micah Elliott
  • 9,600
  • 5
  • 51
  • 54
3

You might use the stdbuf command, like this:

ngrep 'Host:' port 80 -t  -W byline -q | stdbuf -o0 grep '^Host' | tee /tmp/hosts

I'm setting the stdout buffer of grep to zero.

stdbuf is a general hammer that works for programs which are linked against glibc since output/input buffering happens in the libc. The ouput of a program which is writing to a terminal will getting line buffered, the output of a program which is outputting to a pipe or a file will getting block buffered - unless the program itself calls flush(). You can use stdbuf to modify this behaviour.

Btw, ngrep and tee seem to call flush() (ngrep on stdout, tee on stdin) since we didn't needed to use stdbuf for them.

I would accept MicahElliot's answer since --line-buffered is more specific for grep and should be used here. stdbuf is a nice alternative for programs which does not offer such an option (and for guys like me which didn't read grep's man page before ;) )

Community
  • 1
  • 1
hek2mgl
  • 152,036
  • 28
  • 249
  • 266
  • Thanks for recommending which to accept :) I used yours because I'm lazy and it was less typing; I need to go read the man pages for @Micah Elliott's answer to understand the flags! – PeterB Aug 05 '15 at 19:05
  • 1
    @PeterB Btw, MANY THANKS for showing me that nice tool!! I didn't know about `ngrep` before. Cool! A little search gave me also `netsed` - which sounds also promising! – hek2mgl Aug 05 '15 at 19:16