Insert text before the first non-comment-line before a pattern

Question

To update a system configuration file on a Linux server I was in need to add a rule (line of text) before a given other rule (and, if possible, before the comments of that other rule).

With the following input file:

# Foo
# Bar

# Comment about rule on the next line
RULE_A

# Comment about rule on the next line
# Continuation of comment
RULE_B

I want to get the following output:

# Foo
# Bar

# Comment about rule on the next line
RULE_A

# ADDED COMMENT
# ADDED COMMENT CONTINUATION
ADDED_RULE

# Comment about rule on the next line
# Continuation of comment
RULE_B

I ended up with the following combination of :

sed : convert my multi-line text to add in a single line with \n.
tac : to reverse the file.
awk : to work on the file.
A temporary file that will replace the original file (because I don't have "in-place" option on awk)

CONF_FILEPATH="sample.conf"

# Create sample work file:
cat > "${CONF_FILEPATH}" <<EOT
# Foo
# Bar

# Comment about rule on the next line
RULE_1

# Comment about rule on the next line
# Continuation of comment
RULE_2

RULE_3_WITHOUT_COMMENT

RULE_4_WITHOUT_COMMENT
RULE_5_WITHOUT_COMMENT

# Comment about rule on the next line
RULE_6

# Comment about rule on the next line
# Continuation of comment
RULE_7

EOT

# Text (of new rule) to add:
TEXT_TO_ADD="# ADDED COMMENT
# ADDED COMMENT CONTINUATION
ADDED_RULE
"

# The rule before which we want to add our text:
BEFORE_RULE="RULE_7"

# Temporary file:
TMP_FILEPATH="$(mktemp)"

# Convert newlines to \n:
TEXT_TO_ADD_FOR_AWK="$(echo ${TEXT_TO_ADD} | tac | sed -E ':a;N;$!ba;s/\r{0,1}\n/\\n/g')"

# Process
awk 'BEGIN {
    ADD_TO_LINE="";
}
{
    if ($0 ~ "^'${BEFORE_RULE}'") {
        # DEBUG: Got the "deny all" line
        ADD_TO_LINE=NR+1 ;
        print $0;
    } else {
        if (ADD_TO_LINE==NR) {
            # DEBUG: Current line is the candidate
            if ($0 ~ "#") {
                ADD_TO_LINE=NR+1;
                # DEBUG: Its a comment, wont add here, taking note to try on the next line
                print $0;
            } else {
                # DEBUG: Not a comment: this is the place!
                print "'${TEXT_TO_ADD_FOR_AWK}'";
                ADD_TO_LINE="";
                print $0;
            }
        } else {
            print $0;
        }
    }
}' <(tac "${CONF_FILEPATH}") \
    | tac > "${TMP_FILEPATH}"

# Overwrite:
cat "${TMP_FILEPATH}" > "${CONF_FILEPATH}"

# Cleaning up:
rm "${TMP_FILEPATH}"

I then get (look just before RULE_7):

# Foo
# Bar

# Comment about rule on the next line
RULE_1

# Comment about rule on the next line
# Continuation of comment
RULE_2

RULE_3_WITHOUT_COMMENT

RULE_4_WITHOUT_COMMENT
RULE_5_WITHOUT_COMMENT

# Comment about rule on the next line
RULE_6

# ADDED COMMENT
# ADDED COMMENT CONTINUATION
ADDED_RULE

# Comment about rule on the next line
# Continuation of comment
RULE_7

Which is OK, but I'm sure there is a cleaner/simpler way of doing that with awk.

Context: I am editing the /etc/security/access.conf to add an allow rule before the deny all rule.

Your code and the output of it you posted shows that don't always have a blank line or end of file after a RULE line - that's an important use case to include in the sample input/output you provided for us to test with. — Ed Morton, Aug 05 '21 at 19:39

Ed Morton · Answer 1 · 2021-08-05T19:48:10.013

You never need sed when you're using awk:

text_to_add='# ADDED COMMENT
# ADDED COMMENT CONTINUATION
ADDED_RULE
'

before_rule='RULE_B'

awk -v rule="$before_rule" -v text="$text_to_add" '
    /^#/ { cmt = cmt $0 ORS; next }
    $0==rule { print text }
    { printf "%s%s\n", cmt, $0; cmt="" }
' file
# Foo
# Bar

# Comment about rule on the next line
RULE_A

# ADDED COMMENT
# ADDED COMMENT CONTINUATION
ADDED_RULE

# Comment about rule on the next line
# Continuation of comment
RULE_B

If you can have comments after the final non-comment line then just add END { printf "%s", cmt } to the end of the script.

Don't use all-caps variable names (see Correct Bash and shell script variable capitalization) and always quote shell variables (see https://mywiki.wooledge.org/Quotes). Copy/paste your original script into http://shellcheck.net and it'll tell you some of the issues.

Regarding ...because I don't have "in-place" option on awk from your question - GNU awk has -i inplace for that.

score 2 · Answer 2 · answered Aug 05 '21 at 19:33

2

Reading the file paragraph-wise makes things simpler:

awk -v text_to_add="$TEXT_TO_ADD" \
    -v before_rule="$BEFORE_RULE" \
    -v RS='' \
    -v ORS='\n\n' \
'$0 ~ "\n" before_rule {print text_to_add} 1' file

Get out of the habit of using ALLCAPS variable names, leave those as reserved by the shell. One day you'll write PATH=something and then wonder why your script is broken.

answered Aug 05 '21 at 19:33

glenn jackman

238,783
38
220
352

That'd work for the posted sample input/output but if you look at the OPs tool output you'll see cases where there isn't a blank line between RULE lines so it'd fail if the OP wanted to add a text block above `RULE_5_WITHOUT_COMMENT` in that output for example. – Ed Morton Aug 05 '21 at 19:41
1

Ah. A case of "tl;dr" – glenn jackman Aug 05 '21 at 19:43

score 0 · Answer 3 · answered Aug 06 '21 at 01:37

ed, the standard editor, to the rescue! Because it looks at the entire file, not just a line at a time, it's able to move the current line cursor around forwards and backwards with ease:

ed -s input.txt <<EOF
/RULE_7/;?^[^#]?a

# ADDED COMMENT
# ADDED COMMENT CONTINUATION
ADDED_RULE
.
w
EOF

After this, input.txt looks like your desired result.

It first sets the current line to the first one containing RULE_7, then looks backwards for the first non-empty line above it that doesn't start with # (The line with RULE_6 in this case), and appends the desired text after that line. Then it writes the modified file back to disk.

Insert text before the first non-comment-line before a pattern

3 Answers3