General solutions to replace string regex preceded and followed by '\n'

Question

I have a file in CentOS which looks like following

[root@localhost nn]# cat -A excel.log
real1$
0.5^I0.5^I0.5^I1^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I^I0.5^I0.5^I0.5^I1^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I^I0.5^I0.5^I0.5^I1^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I$
real2$
0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I^I0.5^I0.5^I0.5^I1^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I^I0.5^I0.5^I0.5^I1^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I$
real3$
0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I1^I0.5^I0.5^I0.5^I0.5^I^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I1^I0.5^I0.5^I0.5^I0.5^I$
real4$
0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I1^I0.5^I1^I0.5^I0.5^I0.5^I1^I0.5^I0.5^I0.5^I0.5^I$
real5$
0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I1^I0.5^I0.5^I0.5^I0.5^I^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I1^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I0.5^I1^I0.5^I0.5^I0.5^I1^I0.5^I0.5^I0.5^I0.5^I$
real6$

I would like to replace \nreal[2-6]\n with \t\t\t' and have tried unsuccessfully the following

sed -i 's/\nreal[2-6]\n/\t\t\t/g' file

It seems that sed has difficulty to deal with line break. Any idea to fulfill the regex in CentOS?

Much appreciated!

Try `sed -i -z 's/\nreal[2-6]\n/\t\t\t/g' file` if the `sed` is GNU sed. — Wiktor Stribiżew, Oct 31 '20 at 14:51
It seems working for real[2-6], but not working for (512|513)real. — Jonathan Zhou, Oct 31 '20 at 15:23
To be specific, if I want to replace '\n(512|513)real\n' with '\t\t\t', then it won't work. — Jonathan Zhou, Oct 31 '20 at 15:24
`sed -i -Ez 's/\n(512|513)real\n/\t\t\t/g' file`? `sed -i -rz 's/\n(512|513)real\n/\t\t\t/g' file`? `sed -i -z 's/\n51[23]real\n/\t\t\t/g' file`? What is the right answer for you here? What is your goal? — Wiktor Stribiżew, Oct 31 '20 at 15:26
I posted the [`sed` answer](https://stackoverflow.com/a/64623384/3832970). — Wiktor Stribiżew, Oct 31 '20 at 15:31
My goal is that a general regex (i.e. real[2-6] or (512|513)real, specifically in my current file) preceded and followed by '\n' are replaced by '\t\t\t' (or a general regex) with sed. — Jonathan Zhou, Oct 31 '20 at 15:35
Your solution seems to use '?' to join multiple sed, which I don't know if it works. — Jonathan Zhou, Oct 31 '20 at 15:37
I have 3 questions with code inside in the comment + 2 additional questions without code. There are not "multiple seds". — Wiktor Stribiżew, Oct 31 '20 at 15:38
The OR regex is `\n(51[23]real|real[2-6])\n` not the one from the answer selected. — , Oct 31 '20 at 15:54
@Maxt8r: It is because requirements were different originally. Have updated now. Many thanks. — anubhava, Oct 31 '20 at 16:13
Did you try `sed -i -Ez 's/\n(51[23]real|real[2-6])\n/\t\t\t/g' file`? — Wiktor Stribiżew, Oct 31 '20 at 20:06

anubhava · Accepted Answer · 2020-10-31T16:19:15.477

2

If you want to consider perl then use:

perl -i -0777 -pe 's/\n(?:51[23]real|real[2-6])(?:\n|\z)/\t\t\t/g' file

If you want to avoid last real\d+ line to be replaced with \t\t\t then use:

perl -i -0777 -pe 's/\n(?:51[23]real|real[2-6])\n(?!\z)/\t\t\t/g' file

(?!\z) is negative lookahead to fail the match when we have line end just ahead of us.

edited Oct 31 '20 at 16:19

answered Oct 31 '20 at 15:07

anubhava

761,203
64
569
643

This solution works for '\n(512|513)real\n'. – Jonathan Zhou Oct 31 '20 at 15:29
Sorry didn't understand. `\nreal[2-6]\n` is regex which will not match `\n(512|513)real\n` – anubhava Oct 31 '20 at 15:33
If you also want to match 512 or 513 before real then please check my updated answer. – anubhava Oct 31 '20 at 15:40

Wiktor Stribiżew · Answer 2 · 2020-10-31T18:03:26.643

2

With GNU sed, you need to use the -z option:

sed -i -z 's/\nreal[2-6]\n/\t\t\t/g' file
#      ^^

Now, that you also want to handle specific alternations, you need to enable the POSIX ERE syntax, either with -r or -E option:

sed -i -Ez 's/\n(51[23]real|real[2-6])\n/\t\t\t/g' file

edited Oct 31 '20 at 18:03

answered Oct 31 '20 at 15:31

Wiktor Stribiżew

607,720
39
448
563

General solutions to replace string regex preceded and followed by '\n'

2 Answers2