Delete symbol in text file

Question

I have a text file test.txt which I would like to edit by means of a Bash script.

The idea is the read a string from /dev/stdin which is then used to delete all lines beginning with this string. How can I achieve this?

So far I have the following:

#!/bin/bash
cat test.txt
echo "Write del if you want to delete or write save if you want to save"
read s1
echo "Which symbol"
read s2

if [ "$s1" = "save" ]; then
   sed -i '/^'$s2'/!d' test.txt    
elif [ "$s1" = "del" ]; then

fi 
cat test.txt

You would like to delete all lines which start with a particular string. You have a small typo in your `sed` lline. It is `sed -i '/^/'$s2'/d' test.txt`. The exclamation mark does just the inverse. — kvantour, Oct 24 '18 at 17:46
and if you have slashes, then you need to escape them : `sed -i '/^'${s2//\//\\/}'/d' test.txt` (note previous comment has a slash to much (the second one) — kvantour, Oct 24 '18 at 17:50
Wants to delete on the section with the unfinished code. Wants to save (`!d`) on the line provided. — Paul Hodges, Oct 24 '18 at 19:46
Your question has 2 parts: How to use a var in sed (this part is duplicate) and how to match first part of a line without regexpr (looks like https://unix.stackexchange.com/questions/282445/grepping-a-fixed-string-at-the-beginning-of-a-line/282477#282477). I gave an answer to the second question that you might find useful. — Walter A, Oct 25 '18 at 21:46

Paul Hodges · Answer 1 · 2018-10-25T22:40:10.270

update

#!/bin/bash

case "$#" in
2) s1="$1"; s2="$2";;
*) echo "$0 {save|del} <key>"
   exit 1;;
esac

case "$s1" in
save) v=-v ;;
del)  v=   ;;
esac

sed -in "$(cut -c -${#s2} x | grep $v -F -n "$s2" | sed "s/:.*/ d;/" )" test.txt

This handles complex leading metacharacter strings. :)

old stuff

I'm copying test.txt to an editable temp so that I have a consistent source for testing. I also put the args on the commandline because I got tired of typing them separately and wanted to test in a loop.

Yes, I know you were reading stdin, but you had that part working fine and can put it back if you like.

Oh, and I like case statements. :)

#!/bin/bash

case "$#" in
2) s1="$1"; s2="$2";;
*) echo "$0 {save|del} <key>"
   exit 1;;
esac

# make a var that is $s2 but with these metacharacters removed
tst="$( echo "$s2" | tr -d "[/.;\][}{!#^&*)(-?\\\\]" )"
case "$tst" in 
$s2) q=''   ;; # if it's the same, no metas to worry about
  *) q="\\" ;; # else we need to quote the character
esac

cat test.txt >| edited
case "$s1" in
save) sed -in "/^$q$s2/!d" edited ;;
del)  sed -in "/^$q$s2/d"  edited ;;
esac

cat edited

This works even if you use a relevant metacharacter such as / or ., because it quotes it. The code doesn't need to exclude any characters from the list of options.

My source input -

$: cat test.txt
foo1234567890qwertyuiopasdfghjklzxcvbnm
bar1234567890qwertyuiopasdfghjklzxcvbnm
/1234567890qwertyuiopasdfghjklzxcvbnm
&1234567890qwertyuiopasdfghjklzxcvbnm
|1234567890qwertyuiopasdfghjklzxcvbnm
.1234567890qwertyuiopasdfghjklzxcvbnm
?1234567890qwertyuiopasdfghjklzxcvbnm

My test examples:

$: for cmd in save del
   do for k in foo \/ \. \? \& \|
   do echo "
  =====  $cmd $k
   "
   script $cmd "$k"
   done
   done

  =====  save foo

foo1234567890qwertyuiopasdfghjklzxcvbnm

  =====  save /

/1234567890qwertyuiopasdfghjklzxcvbnm

  =====  save .

.1234567890qwertyuiopasdfghjklzxcvbnm

  =====  save ?

?1234567890qwertyuiopasdfghjklzxcvbnm

  =====  save &

&1234567890qwertyuiopasdfghjklzxcvbnm

  =====  save |

|1234567890qwertyuiopasdfghjklzxcvbnm

  =====  del foo

bar1234567890qwertyuiopasdfghjklzxcvbnm
/1234567890qwertyuiopasdfghjklzxcvbnm
&1234567890qwertyuiopasdfghjklzxcvbnm
|1234567890qwertyuiopasdfghjklzxcvbnm
.1234567890qwertyuiopasdfghjklzxcvbnm
?1234567890qwertyuiopasdfghjklzxcvbnm

  =====  del /

foo1234567890qwertyuiopasdfghjklzxcvbnm
bar1234567890qwertyuiopasdfghjklzxcvbnm
&1234567890qwertyuiopasdfghjklzxcvbnm
|1234567890qwertyuiopasdfghjklzxcvbnm
.1234567890qwertyuiopasdfghjklzxcvbnm
?1234567890qwertyuiopasdfghjklzxcvbnm

  =====  del .

foo1234567890qwertyuiopasdfghjklzxcvbnm
bar1234567890qwertyuiopasdfghjklzxcvbnm
/1234567890qwertyuiopasdfghjklzxcvbnm
&1234567890qwertyuiopasdfghjklzxcvbnm
|1234567890qwertyuiopasdfghjklzxcvbnm
?1234567890qwertyuiopasdfghjklzxcvbnm

  =====  del ?

foo1234567890qwertyuiopasdfghjklzxcvbnm
bar1234567890qwertyuiopasdfghjklzxcvbnm
/1234567890qwertyuiopasdfghjklzxcvbnm
&1234567890qwertyuiopasdfghjklzxcvbnm
|1234567890qwertyuiopasdfghjklzxcvbnm
.1234567890qwertyuiopasdfghjklzxcvbnm

  =====  del &

foo1234567890qwertyuiopasdfghjklzxcvbnm
bar1234567890qwertyuiopasdfghjklzxcvbnm
/1234567890qwertyuiopasdfghjklzxcvbnm
|1234567890qwertyuiopasdfghjklzxcvbnm
.1234567890qwertyuiopasdfghjklzxcvbnm
?1234567890qwertyuiopasdfghjklzxcvbnm

  =====  del |

foo1234567890qwertyuiopasdfghjklzxcvbnm
bar1234567890qwertyuiopasdfghjklzxcvbnm
/1234567890qwertyuiopasdfghjklzxcvbnm
&1234567890qwertyuiopasdfghjklzxcvbnm
.1234567890qwertyuiopasdfghjklzxcvbnm
?1234567890qwertyuiopasdfghjklzxcvbnm

This still leaves the problem of the case where the leading string consists of multiple metacharacters... I'm thinking on that one...

So, alternate version:

#!/bin/bash

case "$#" in
2) s1="$1"; s2="$2";;
*) echo "$0 {save|del} <pattern>"
   exit 1;;
esac

cat test.txt >| edited
case "$s1" in
save) act="!d" ;;
del)  act="d"  ;;
esac
sed -in "$s2$act"  edited

cat edited

This means you have to put the search pattern in on the command line (correctly), but then you can handle any weirdness you like. For example, I added a line -

/.?1234567890qwertyuiopasdfghjklzxcvbnm

and ran it as script save "/^[/][.][?]/", which worked fine.

The alternate version requires the user to write valid `sed` syntax. It will work for a smart user. — Walter A, Oct 24 '18 at 22:14
Special characters makes it difficult. Perhaps use domething like https://unix.stackexchange.com/a/282477/57293 . — Walter A, Oct 24 '18 at 22:21
That's good, but requires line-by-line processing. I was trying to avoid that since he didn't do it in the OP, but I do like that option. Also allows `case "$line" in; $s2*) : code here;; esac` ;) — Paul Hodges, Oct 25 '18 at 13:34
I first couldn't find a solution without a loop, so I did not post an answer. Finally I thought of https://unix.stackexchange.com/a/477845/57293 what would be useful here too. — Walter A, Oct 25 '18 at 21:48

score 0 · Answer 2 · edited Oct 24 '18 at 18:15

0

I have read through some of the comments and decides to revise my answer. My new answer is as follows:

sed '/^'${s2//\//\\/}'/d'

Based on kvantour comments this is also a viable solution as well:

sed -i '/^'${s2//\//\\/}'/d' test.txt

original answer: (see comments)

So it is because of way you are referencing the variable aka the puncation around it, I got a few ideas that you could try instead. Let me know if they any of them work or not:

1.sed '/^$s2/ d'
2.sed "/^$s2/ d"
3.sed "/^'$s2'/ d"
4.sed -i "|^'$s2'| d" test.txt

edited Oct 24 '18 at 18:15

kvantour

25,269
4
47
72

answered Oct 24 '18 at 17:44

B. Cratty

1,725
1
17
32

@kvantour what about my updated option 3 and option 4? – B. Cratty Oct 24 '18 at 17:50
@kvantour thank you, it was not rude. I actually enjoed that. The information you gave was very beneficial to me and most likely to the user who posted this question as well for future uses. – B. Cratty Oct 24 '18 at 18:02
1

Regarding your first case you have a problem that bash will not perform any substitution within single quoted strings. You can see this when you write `foo="test" echo "$foo"` and `foo="test" echo '$foo'`. You will see that the second will not write `test` – kvantour Oct 24 '18 at 18:05
1

Regarding your second case, This is a good case but has a single problem. What if the symbol is a . This would result in `sed "/^//d"`. this is an invalid sed command. You would have to escape the slash as `sed "/^\//d"`. So you would have to perform a bash substitution to change all `/` into `\/`. You can do this with `${s2//\//\\/}` – kvantour Oct 24 '18 at 18:07
1

Regarding your third case, this is not in line with what the OP requested. Here you will delete all lines which start with a single-quoted string `$s2`. The single quotes are not valid. So i would remove this case – kvantour Oct 24 '18 at 18:09
1

Regarding your fourth case. Here you have a common confusion going on. sed commands are written as `address function`. Most people know the substitution function `s/BRE/replacement/flags` and know that you can replace the slashes with any other character. This is then also the only case where this is valid. Address specifiers are mostly line numbers or regex matches as `/string/` These regex addresses are always with slashes, no exception. The function we use here is `d`. So you cannot use the pipe as a replacement for the slashes. – kvantour Oct 24 '18 at 18:11
@kvantour Sincerely thank you for the information. I gathered all those previous cases from other posts and other research I was doing. I have edited my answer to fit with the comments you have given as well as provided the answer you gave the user in the original post's comment section. – B. Cratty Oct 24 '18 at 18:14
You're welcome. I have updated your answer to include your original answer. This way other people can learn from this. – kvantour Oct 24 '18 at 18:16
Most metacharacters are problematic. What if you want to keep or delete lines that start with a dot(`.`)? See my answer below, I think it handles it reasonably well. – Paul Hodges Oct 24 '18 at 19:44
How about `s2="a b"` of `s2="a(b"` ? – Walter A Oct 24 '18 at 22:11

Delete symbol in text file

2 Answers2