3

I want to replace $foo with bar. This works fine.

$ echo '$foo' | sed 's/$foo/bar/'
bar

But this command does not work fine when I use the -r option.

$ echo '$foo' | sed -r 's/$foo/bar/'
$foo

Why doesn't this work.

Here is an example of what works with -r option.

$ echo '$foo' | sed -r 's/\$foo/bar/'
bar

The real question is: Why does $ need to be escaped only while using the -r option. What would $ mean otherwise with the -r option?

I am using a Debian Linux system.

Lone Learner
  • 18,088
  • 20
  • 102
  • 200
  • You don't want to expand the `$foo` variable on using `-r`? – Inian Jun 30 '16 at 04:48
  • `echo '$foo' | sed -r 's/\$foo/bar/'`? – Cyrus Jun 30 '16 at 05:06
  • $foo will be expanded by shell when you use double quotation mark `" "`. – gzh Jun 30 '16 at 05:24
  • @gzh The `$` in `$foo` is already escaped with `\$`, so it should not be escaped. Try `echo "\$foo"`. – Lone Learner Jun 30 '16 at 05:28
  • See: [Difference between single and double quotes in bash](http://stackoverflow.com/q/6697753/3776858) – Cyrus Jun 30 '16 at 05:30
  • @Cyrus What you are suggesting is equivalent to `echo '$foo' | sed -r "s/\\\$foo/bar/"`. Yes, it works. That brings the question why `$` need to be escaped in the regex (not in the shell). What does `$` not at the end of the pattern mean when `-r` option is specified? – Lone Learner Jun 30 '16 at 05:31
  • @Cyrus I understand the difference between single and double quotes already. If you think double-quotes is the problem, then why does the first regular expression in my question works? – Lone Learner Jun 30 '16 at 05:32
  • @LoneLearner, bash will expand $foo, then transfer the expanded result to sed command. – gzh Jun 30 '16 at 05:35
  • @Cyrus Since you guys are unnecessarily bothering about double-quotes in this question, I have rewritten the question using single-quotes only. The problem is still evident. – Lone Learner Jun 30 '16 at 05:36
  • @gzh I have edited the question to use single-quotes only. Now you would agree that bash would not expand the variable `$foo`, right? But the problem is still there. See the updated question. – Lone Learner Jun 30 '16 at 05:37
  • What is "this command does not work fine" vs "this works fine"? "fine" meaning what exactly? – l'L'l Jun 30 '16 at 05:40
  • @l'L'l Read the first sentence of this question. – Lone Learner Jun 30 '16 at 05:41

1 Answers1

4

In extended regular expressions (ERE), that is, under -r, $ means the end of the line:

$ echo '$foo' | sed -r 's/foo$/bar/'
$bar

If you want it to mean something else, it has to be escaped:

$ echo '$foo' | sed -r 's/[$]foo$/bar/'
bar

Documentation

man 7 regex explains that, in Extended Regular Expressions (ERE), the $ matches at the end of the line:

'$' (matching the null string at the end of a line)

The same man page goes on to explain that in Basic Regular Expressions (BRE), which is what you get without -r, its meaning is more complicated:

Obsolete ("basic") regular expressions differ in several respects. '|', '+', and '?' are ordinary characters and there is no equivalent for their functionality. The delimiters for bounds are "{" and "}", with '{' and '}' by themselves ordinary characters. The parentheses for nested subexpressions are "(" and ")", with '(' and ')' by themselves ordinary characters. '^' is an ordinary character except at the beginning of the RE or(!) the beginning of a parenthesized subexpression, '$' is an ordinary character except at the end of the RE or(!) the end of a parenthesized subexpression, and '*' is an ordinary character if it appears at the beginning of the RE or the beginning of a parenthesized subexpression (after a possible leading '^').

John1024
  • 109,961
  • 14
  • 137
  • 171
  • Do you know why `$` would mean the end of the line even when it is not at the end of the pattern? In basic regular expression, `$` matches end of line only when it is at the end of the pattern. Why is it different for extended regular expression? How could `$` at the beginning of a pattern match an end of line anyway? – Lone Learner Jun 30 '16 at 05:45
  • @LoneLearner I just added the manpage documentation for it. As for why, that is not clear to me: it seems like a poor design choice. – John1024 Jun 30 '16 at 05:48
  • Thanks for quoting the documentation. – Lone Learner Jun 30 '16 at 05:49