Grep a line from a file and replace a substring and append the line to the original file in bash?

Question

This is what I want to do.

for example my file contains many lines say :

ABC,2,4
DEF,5,6
GHI,8,9

I want to copy the second line and replace a substring EF(all occurrences) and make it XY and add this line back to the file so the file looks like this:

ABC,2,4
DEF,5,6
GHI,8,9
DXY,5,6

how can I achieve this in bash?

EDIT : I want to do this in general and not necessarily for the second line. I want to grep EF, and do the substition in whatever line is returned.

Answer 1

Here's a simple Awk script.

awk -F, -v pat="EF" -v rep="XY" 'BEGIN { OFS=FS }
  $1 ~ pat { x = $1; sub(pat, rep, x); y = $0; sub($1, x, y); a[++n] = y }
  1
  END { for(i=1; i<=n; i++) print a[i] }' file

The -F , says to use comma as the input field separator (internal variable FS) and in the BEGIN block, we also set that as the output field separator (OFS).

If the first field matches the pattern, we copy the first field into x, substitute pat with rep, and then substitute the first field of the whole line $0 with the new result, and append it to the array a.

1 is a shorthand to say "print the current input line".

Finally, in the END block, we output the values we have collected into a.

This could be somewhat simplified by hardcoding the pattern and the replacement, but I figured it's more useful to make it modular so that you can plug in whatever values you need.

While this all could be done in native Bash, it tends to get a bit tortured; spending the 30 minutes or so that it takes to get a basic understanding of Awk will be well worth your time. Perhaps tangentially see also while read loop extremely slow compared to cat, why? which explains part of the rationale for preferring to use an external tool like Awk over a pure Bash solution.

Answer 2

You can use the sed command:

sed '
/EF/H # copy all matching lines
${    # on the last line
  p   # print it
  g   # paste the copied lines
  s/EF/XY/g # replace all occurences
  s/^\n//   # get rid of the extra newline
}'

As a one-liner:

sed '/EF/H;${p;g;s/EF/XY/g;s/^\n//}' file.csv

Answer 3

If ed is available/acceptable, something like:

#!/bin/sh

ed -s file.txt <<-'EOF'
  $kx
  g/^.*EF.*,.*/t'x
  'x+;$s/EF/XY/
  ,p
  Q
EOF

Or in one-line.

printf '%s\n' '$kx' "g/^.*EF.*,.*/t'x" "'x+;\$s/EF/XY/" ,p Q | ed -s file.txt

---

• Change Q to w if in-place editing is needed.

• Remove the ,p to silence the output.

Answer 4

Using BASH:

#!/bin/bash

src="${1:-f.dat}"
rep="${2:-XY}"

declare -a new_lines
while read -r line ; do
  if [[ "$line" == *EF* ]] ; then
    new_lines+=("${line/EF/${rep}}")
  fi
done <"$src"

printf "%s\n" "${new_lines[@]}" >> "$src" 

Contents of f.dat before:

ABC,2,4
DEF,5,6
GHI,8,9

Contents of f.dat after:

ABC,2,4
DEF,5,6
GHI,8,9
DXY,5,6

Answer 5

Following on from the great answer by @tripleee, you can create a variation that uses a single call to sub() by outputting all records before the substitution is made, then add the updated record to the array to be output with the END rule, e.g.

awk -F, '1; /EF/ {sub(/EF/,"XY"); a[++n]=$0} END {for(i=1;i<=n;i++) print a[i]}' file

Example Use/Output

An expanded input based on your answer to my comment below the question that all occurrences of EF will be replaced with XY in all records, e.g.

$ cat file
ABC,2,4
DEF,5,6
GHI,8,9
EFZ,3,7

Use and output:

$ awk -F, '1; /EF/ {sub(/EF/,"XY"); a[++n]=$0} END {for(i=1;i<=n;i++) print a[i]}' file
ABC,2,4
DEF,5,6
GHI,8,9
EFZ,3,7
DXY,5,6
XYZ,3,7

Let me know if you have questions.