Linux bash: grep from stream and write to file

Question

I have a log file A that is constantly updated (but it is rolled over) and I need to constantly filter it's content and write to a persistent file.

TL;DR I need to:

tail -f A.log | grep "keyword" >> B.log

But this command does not write anything to B.log.

Research only got me complex stuff that is not my case. My guess is that I'm missing some simple concept.

This is not the same question marked as possible duplicate, as the grep works and I have it's output if I don't try to write it to a file. The problem is the file.

@WebertS.Lima The output to file will work if `--line-buffered` is included — Leon, Aug 11 '16 at 12:02
@Leon oh, you're right. Post it as an answer and I'll accept it. — Webert Lima, Aug 11 '16 at 12:15
@WebertS.Lima rather up vote the answer on the related question +1 — Leon, Aug 11 '16 at 12:16

hek2mgl · Accepted Answer · 2016-08-11T12:27:55.220

3

If just grep, without writing to the file, works, you encountered a buffering "problem". I/O buffering, unless manually implemented by the program will get handled by the libc. If the program's stdout is a termial, buffering will be line-based. If not, the libc buffers output until the buffer reached a size limit.

On Linux, meaning with glibc you can use the stdbuf command to configure that buffering:

tail -f A.log | stdbuf -oL grep "keyword" >> B.log

-oL specifies that the output stream should be line-buffered.

edited Aug 11 '16 at 12:27

answered Aug 11 '16 at 12:02

hek2mgl

152,036
28
249
266

That works too. Thank you. – Webert Lima Aug 11 '16 at 12:21
@WebertS.Lima While `--line-buffered` is specific to `grep`, `stdbuf` can be used for any program which is linked against glibc and leaves buffering to the libc. – hek2mgl Aug 11 '16 at 12:24
noted! =) I'll keep that in mind. – Webert Lima Aug 11 '16 at 12:27
@WebertS.Lima Uh, I need to correct me. Since `--line-buffered` is not POSIX compatible, it is not portable as well. In that case `stdbuf` is just fine. – hek2mgl Aug 11 '16 at 12:28
I see. Thank you very much for your concern. – Webert Lima Aug 11 '16 at 14:32

Linux bash: grep from stream and write to file

1 Answers1