Removing a delimited block of lines when one of them matches a regex pattern with awk

Question

Let's assume the following reprepro distributions file:

Origin: git.sdxlive.com/git/PPA
Label: Ubuntu focal
Suite: focal
Version: 20.04
Codename: focal
Architectures: i386 amd64
Components: stable unstable
Limit: 0
Description: Latest Ubuntu focal 20.04 packages
Contents: .gz .bz2
Tracking: keep
SignWith: xxxxxxxxxxxxxxxxxxxx
Signed-By: xxxxxxxxxxxxxxxxxxxx
ValidFor: 2y 6m
Log: packages.Ubuntu.log

Origin: git.sdxlive.com/git/PPA
Label: Ubuntu groovy
Suite: groovy
Version: 20.10
Codename: groovy
Architectures: i386 amd64
Components: stable unstable
Limit: 0
Description: Latest Ubuntu groovy 20.10 packages
Contents: .gz .bz2
Tracking: keep
SignWith: xxxxxxxxxxxxxxxxxxxx
Signed-By: xxxxxxxxxxxxxxxxxxxx
ValidFor: 2y 6m
Log: packages.Ubuntu.log

The goal is to remove the whole block of lines delimited by 'Origin: ' and an empty line when it contains the line "Codename: ${os_code_name}" where os_code_name is a bash variable.

So the expected output is:

Origin: git.sdxlive.com/git/PPA
Label: Ubuntu groovy
Suite: groovy
Version: 20.10
Codename: groovy
Architectures: i386 amd64
Components: stable unstable
Limit: 0
Description: Latest Ubuntu groovy 20.10 packages
Contents: .gz .bz2
Tracking: keep
SignWith: xxxxxxxxxxxxxxxxxxxx
Signed-By: xxxxxxxxxxxxxxxxxxxx
ValidFor: 2y 6m
Log: packages.Ubuntu.log

Without a variable Codename, we could use for instance the following to remove the block matching the focal Codename:

awk '/^Origin: /{s=x} {s=s $0 RS} /^$/{if(s!~/Codename: focal/) printf "%s",s}' distributions

I could not find a solution to use a variable Codename; I tried to use:

--assign=var="${os_code_name}"
ENVIRON["os_code_name"]

In the first case, I don't know how awk can differentiate between the string 'Codename: ' and the variable var, since we cannot use "$var". The following does not work obviously:

awk --assign=var="${os_code_name}" '/^Origin: /{s=x} {s=s $0 RS} /^$/{if(s!~/Codename: $var/) printf "%s",s}' distributions

In the second case, it is also unsuccessful:

awk '/^Origin: /{s=x} {s=s $0 RS} /^$/{if(s!~/Codename: ENVIRON["os_code_name"]/) printf "%s",s}' distributions

I also checked this answer.

Any suggestion?

Good that you have shown your efforts in form of code keep it up. Could you please also post samples of expected output in your question and let us know then for more clarity — RavinderSingh13, Sep 12 '20 at 17:53

RavinderSingh13 · Accepted Answer · 2020-09-12T18:28:02.990

Could you please try following, written and tested with shown samples and should work in all kind of awk.

os_code_name="focal" ##shell variable
awk -v co="$os_code_name" '
/Origin/{
  if(!foundCo && FNR>1){ print val }
  val=foundCo=""
}
/^Codename/ && $NF==co{
  foundCo=1
}
{
  val=(val?val ORS:"")$0
}
END{
  if(!foundCo){ print val }
}
'  Input_file

Explanation: Adding detailed explanation for above.

os_code_name="focal"                     ##This is a shell variable.
awk -v co="$os_code_name" '              ##Starting awk program from here and setting co variable as value of os_code_name here.
/Origin/{                                ##Checking condition if line has Origin string in it then do following.
  if(!foundCo && FNR>1){ print val }     ##Checking condition if foundCo is NULL and FNR>1 then print val here.
  val=foundCo=""                         ##Nullifying variables here.
}
/^Codename/ && $NF==co{                  ##Checking condition if line starts with Codenam and last field is equal to variable.
  foundCo=1                              ##Setting value for foundCo here.
}
{
  val=(val?val ORS:"")$0                 ##Creating val which has all lines values from Origin to just before next occurrence of Origin it either gets printed above or gets NULL.
}
END{                                     ##Starting END block of this awk program from here.
  if(!foundCo){ print val }              ##Checking condition if foundCo is NULL then print val here.
}
'  Input_file                            ##Mentioning Input_file name here.

thanasisp · Answer 2 · 2020-09-12T18:19:25.513

1

You can use empty RS, this is the paragraph mode, and do not print any record where that codename exists.

awk -v cn="$cn" -v RS="" '!($0 ~ "Codename: " cn){print $0,"\n"}' file

The variable has to be passed in the way your linked answer says. The pattern matching can be done either using ~ /.../ or ~ "...", using the double quotes is what you have to do here, "Codename: " var is the matching string.

edited Sep 12 '20 at 18:19

answered Sep 12 '20 at 18:09

thanasisp

5,855
3
14
31

I like its simplicity but the delimiting line has been swallowed, meaning that this is an issue: if there is a third block, the output displays only one block without any empty delimiting line anymore bewteen the 2 remaining blocks. – jean-christophe manciot Sep 12 '20 at 18:24
1

I have already added a print statement for that reason, tested with more blocks also. – thanasisp Sep 12 '20 at 18:25

Removing a delimited block of lines when one of them matches a regex pattern with awk

2 Answers2