How to ignore line in shell script and print only required lines

Question

Thank you for looking into my concern!

I have developed a script run.sh which is creating a output.txt file and storing output in that file. But the output has multiple unwanted lines.. I want script to ignore those lines and only paste the required lines-

Output-

*
*
*
adhkjhajsdkjhas
asdjkakjdhajskh
HSHDKJ
jdkjadjk
sajkdhakjh
ajhdjs
*
*
*
     A565253662     Value
1.   abc           01Nov2019
2.   cde           03Dec2019
3.   def           04Feb2020
4.   hij           16Aug2021
5.   klm           07Dec2021
*
*
*
     A565253665     Value
1.   abc           23Sep2019
2.   cde           24Sep2019
3.   def           01Aug2020
4.   hij           09Sep2021
5.   klm           23Sep2021
*
*
*

But I want output.txt to only print following output instead of above output-

     A565253662     Value
1.   abc           01Nov2019
2.   cde           03Dec2019
3.   def           04Feb2020
4.   hij           16Aug2021
5.   klm           07Dec2021
     A565253665     Value
1.   abc           23Sep2019
2.   cde           24Sep2019
3.   def           01Aug2020
4.   hij           09Sep2021
5.   klm           23Sep2021

Note- Column names are not same always, and

*
*
*
adhkjhajsdkjhas
asdjkakjdhajskh
HSHDKJ
jdkjadjk
sajkdhakjh
ajhdjs
*
*
*

these lines are sometimes capital too.

I have tried read, foreach and line, but maybe using them wrong as I am new to shell.. If anyone has any solution over it.. please help.. PS Thanks in advance :)

use **egrep**: `cat file.txt | egrep -v '/^\*/'` – DiD Sep 27 '21 at 11:44 — DiD, Sep 27 '21 at 11:44

William Pursell · Answer 1 · 2021-09-27T12:43:20.277

You haven't really provided enough information, but if you want to print lines in which the second column is the text "Value" and the 5 subsequent lines, you could simply do:

awk '$2 == "Value"{a = 6} a-- > 0'

Whenever a line in which the 2nd column is "Value" is seen, this sets a counter (the variable a). It then decrements that counter on each subsequent line, printing the line while the counter remains greater than 0.

But, if you don't merely want to print the 5 lines after 'Value', but just want to print all the lines that begin with a digit, you could do:

awk 'a && ! /^[0-9]/ { a = 0 } $2 == "Value" { a = 1} a'

If you want to keep printing until you see a line that starts with a literal asterisk, you could do:

awk 'a && /^\*/ { a = 0 } $2 == "Value" { a = 1} a'

Or perhaps you just want to print lines that have more than 1 column:

awk 'NF > 1'

You need to accurately characterize the lines that you want to print.

Once you've decided how to characterize the lines you want to keep, you can either do run.sh | awk ..., or continue to write the output of run.sh to output.txt and then filter the file with awk ... output.txt (eg, run.sh | awk 'NF>1' or awk 'NF>1' output.txt

score 0 · Answer 2 · answered Sep 27 '21 at 11:45

You can use grep, searching for the strings that will always be in your desired output, like this:

cat output.txt | grep '\.\|Value'

If output.txt is:

*
*
*
adhkjhajsdkjhas
asdjkakjdhajskh
HSHDKJ
jdkjadjk
sajkdhakjh
ajhdjs
*
*
*
     A565253662     Value
1.   abc           01Nov2019
2.   cde           03Dec2019
3.   def           04Feb2020
4.   hij           16Aug2021
5.   klm           07Dec2021
*
*
*
     A565253665     Value
1.   abc           23Sep2019
2.   cde           24Sep2019
3.   def           01Aug2020
4.   hij           09Sep2021
5.   klm           23Sep2021
*
*
*

It will give you:

     A565253662     Value
1.   abc           01Nov2019
2.   cde           03Dec2019
3.   def           04Feb2020
4.   hij           16Aug2021
5.   klm           07Dec2021
     A565253665     Value
1.   abc           23Sep2019
2.   cde           24Sep2019
3.   def           01Aug2020
4.   hij           09Sep2021
5.   klm           23Sep2021

score 0 · Answer 3 · answered Sep 27 '21 at 11:47

0

Grep it

$ grep -Ei '[0-9]*\.|value' file
     A565253662     Value
1.   abc           01Nov2019
2.   cde           03Dec2019
3.   def           04Feb2020
4.   hij           16Aug2021
5.   klm           07Dec2021
     A565253665     Value
1.   abc           23Sep2019
2.   cde           24Sep2019
3.   def           01Aug2020
4.   hij           09Sep2021
5.   klm           23Sep2021

And if you want to save it like this than put this grep in pipe:

your_command_to_generate_log | grep -Ei '[0-9]*\.|value' > output.txt

answered Sep 27 '21 at 11:47

Ivan

6,188
1
16
23

Hi @Ivan and @javiervidrua thank you for your answers but both the answers are working only for half the part, as I forgot to mention the column names are not common always.. also it is not ignoring the first few lines which are ```adhkjhajsdkjhas asdjkakjdhajskh HSHDKJ jdkjadjk sajkdhakjh ajhdjs``` for some reason and only printing this output – Saurabh Sep 27 '21 at 12:16
``` asdjkakjdhajskh HSHDKJ jdkjadjk sajkdhakjh ajhdjs A565253662 Value 1. abc 01Nov2019 2. cde 03Dec2019 3. def 04Feb2020 4. hij 16Aug2021 5. klm 07Dec2021 A565253665 Value ``` – Saurabh Sep 27 '21 at 12:17

How to ignore line in shell script and print only required lines

3 Answers3