0

I am new to shell scripting and trying to grep some text in multiple gz files using awk. My code
zcat log*.gz | awk{awk logic goes here}

but the above takes a lot of time to sift through prod logs.Is there any way to make it run faster?

Java
  • 171
  • 2
  • 3
  • 11
  • Does this answer your question? [Use zcat and sed or awk to edit compressed .gz text file](https://stackoverflow.com/questions/28567685/use-zcat-and-sed-or-awk-to-edit-compressed-gz-text-file) – Adam Katz Mar 13 '20 at 18:30

2 Answers2

0

You might see some output faster if you loop over the files instead of uncompressing them all at once:

for log in log*.gz; do
    zcat "$log" | awk 'awk logic goes here'
done
glenn jackman
  • 238,783
  • 38
  • 220
  • 352
0

I wrote a script called zawk which can do this natively. It's similar to glenn jackman's answer but it handles awk options and several different compression mechanisms and input methods while retaining FILENAME and FNR.

You'd use it like:

zawk 'awk logic goes here' log*.gz
Adam Katz
  • 14,455
  • 5
  • 68
  • 83