7

I am trying to understand this peculiar behavior. Basically, I'm trying to grep an output of a command while still keeping the first line/header. Thanks for the help in advance.

Success Case

ps -ef | { head -1; grep bash; }

Output: 

UID        PID  PPID  C STIME TTY          TIME CMD
username  1008     1  0 Jan21 tty1     00:00:00 -bash
username  1173  1008  0 Jan21 tty1     00:00:00 -bash

Failed Case

 ls -tlrh / | { head -1; grep tmp; }

Output: 

total 100K

(i.e.: it ignores the /tmp folder)

Kent
  • 189,393
  • 32
  • 233
  • 301
kuronue
  • 306
  • 2
  • 8
  • 2
    so far the two answers solved the problem, but I think OP wanted to understand why one case was success and the other one failed. The two answers cannot explain it. I edited the question by adding several tags. I do group command then pipe to another cmd, but not tested to pipe to command group. Interesting question +1 want to know the reason. BTW, if you check the 2nd failed case `echo $?` the return value is `1`, but I cannot see any error msg by adding `2>&1` to all commands... – Kent Jan 22 '15 at 10:52
  • I can't replicate this on my system... Is there possibly some sort of permissions issue preventing a non-root user from reading `/` or something? – twalberg Jan 22 '15 at 15:20
  • @Kent. Thanks for the edit and yes, I would like to understand the reason behind it. I noticed it's not limited to these 2 commands only. I used check_mk monitoring and the 'cmk -D' ouput gave the success result as well. – kuronue Jan 23 '15 at 02:25

5 Answers5

6

@Jotne's answer is better, but sometimes you can use grep -E if you know something in the first line, then you can search for that OR the other thing you want like this with the pipe symbol to express the alternation:

ps -ef | grep -E "UID|bash"

Output

UID   PID  PPID   C STIME   TTY           TIME CMD
502   510   509   0  8:01am ttys000    0:00.08 -bash
502 48806   510   0 10:18am ttys000    0:00.00 grep -E UID|bash
Mark Setchell
  • 191,897
  • 31
  • 273
  • 432
  • Thanks for the answer. Yah, this is the way that I've been using but I dont really like it because I need to look into the output first just to get the line. – kuronue Jan 23 '15 at 02:32
  • This is actually nice for casual use. Most times you want something fast and not type too much – Cosmin Lehene Jun 17 '15 at 18:05
4

Try use awk, eks:

ls -tlrh / | awk 'NR==1 || /tmp/'

This will print line number 1 or lines with tmp

NR==1; print line number 1
/tmp/ print all lines that contains tmp

Jotne
  • 40,548
  • 12
  • 51
  • 55
  • Thanks for the solution. Its very elegant and I like it. However, I cannot mark it as answer since it doesnt explain the reason for the peculiar behaviour. – kuronue Jan 23 '15 at 02:31
  • 1
    The first line will be printed twice if it matches the regex. Combine the conditions using OR instead of using two awk commands: `awk 'NR==1 || /tmp/'` – pabouk - Ukraine stay strong May 05 '22 at 17:21
1

The reason this does not work, is that the first of the two processes (head -n1) reads more than it outputs. It eats up the output of ls and leaves nothing for the grep process; ps creates its output linewise.

The correct way to solve this would be to duplicate STDOUT for every process that needs it, as described here

redirect COPY of stdout to log file from within bash script itself

However here it would suffice to simply feed the reading scripts line by line to avoid any buffering issues:

ls -ltrh / | while { read a; } do echo $a; done | { head -n 1; grep tmp; }

However, this means that grep can not see the line(s), head has consumed.

Community
  • 1
  • 1
nlu
  • 1,903
  • 14
  • 18
  • Thanks for the info. I tried to run the command but I still get the same output as the original command I ran. Or, am I missing something? – kuronue Jan 23 '15 at 02:21
  • This worked for me; however if it does not for you, you should stick to the canonical solution mentioned (dupping stdout). This would be the one outlined below by Zoltán Nagy - tee >(command >&3) duplicates the output to the first command, sending this commands output to fd 3, then grep uses stdin and in the end the output of fd 3 is redirected to stdout. – nlu Jan 23 '15 at 11:06
  • I do not see a reason why would the inserted script `while read ...` change the input buffering of `head` to line buffering. The inserted script can certainly process multiple lines and send them to `stdout` together in a single `write()` call. – pabouk - Ukraine stay strong Apr 22 '22 at 10:04
0

well.. this works to...but just for noting a very convoluted solution

(ps aux | tee >(head -n1 >&3 ) | grep bio >&3 ) 3>&1

this is not as nice as i wanted it to be, fd3 usage makes it weird

note: it might even theorecally possible that grep output preceeds the header

Zoltán Haindrich
  • 1,788
  • 11
  • 20
  • Great idea allowing to use real `grep` or any other filter! --- Unfortunately the implementation does not work. When `head` stops reading (soon after first line), `tee` terminates and rest of the output is not processed. Also the first line can show twice if it matches in `grep`. This fixes the problems: `(ps aux | tee >(head -n1 >&3 ; cat >/dev/null) | tail -n+2 | grep bio >&3 ) 3>&1` --- Not sure how to nicely fix possible grep output preceding head output. To make the chance negligible grep in a sub-shell can be preceded by `sleep .1` and/or run with a lower priority: `nice grep bio`. – pabouk - Ukraine stay strong May 05 '22 at 17:54
  • I made an answer by improving your idea (using `awk`) here: [How to grep a specific line _and_ the first line of a file?](https://unix.stackexchange.com/a/701574/19702) – pabouk - Ukraine stay strong May 06 '22 at 21:20
0

pure sed solution ;)

ps aux|sed '1p;/kwork/p;d'

Zoltán Haindrich
  • 1,788
  • 11
  • 20