What is a "string-ending newline" in relation to the $ metacharacter?

Question

I'm studying regular expressions from Mastering Regular Expressions, 3rd Edition, and I've come across the statement that $ is a bit more complex than ^, which surprised me as I thought they were "symmetrical", except when they are escaped to mean their literal counterparts.

In fact, at page 129, their description is slightly different, with more words spent in favour of $; however I'm still confused about it.

• Regarding ^, only two clear alternatives are described:

Caret ^ matches at the beginning of the text being searched, and, if in an enhaced line-anchor mode, after any newline. [...] $ [...] matches

• As regards $, the description is more obscure to me:

$ [...] matches at the end of the target string, and before a string-ending newline, as well. The latter is common, to allow an expression like s$ (ostensibly, to match "a line ending with s") to match …s<NL>, a line ending with s that's capped with an ending newline.

Two other common meanings for $ are to match only at the end of the target text, and to match before any newline.

The latter two meanings seem pretty symmetric to those described for ^, but what about the string-ending newline meaning?

Searching for [regex] "string-ending newline" only gives one, two, and three results, at the moment, and all of them refer to

$ Matches the ending position of the string or the position just before a string-ending newline. In line-based tools, it matches the ending position of any line.

Answer 1

Zero width assertion $ asserts position at the end of the string, or before the line terminator right at the end of the string (if any).

It will be more clear with these code snippets in perl:

$str = 'abc
foo';
$str =~ s/\w+$/#/;
print "1. <" . $str . ">\n\n";

$str = 'abc
foo
';
$str =~ s/\w+$/#/;
print "2. <" . $str . ">\n\n";

$str = 'abc
foo

';
$str =~ s/\w+$/#/;
print "3. <" . $str . ">\n\n";

This will generate this output:

1. <abc
#>

2. <abc
#
>

3. <abc
foo

>

As you can see that $ matches cases 1 and 2 because $ matches at the end of string (case 1) or before the line break right at the end (case 2). However case 3 remains unmatched because line break is not at the end of string.

Answer 2

"String-ending newline" refers to a line feed that is the last character of a string.

---

Without /m

$ matches before a line feed at the end of the string, and at the end of the string.

"abc\ndef\n" =~ /^abc$/           # Doesn't match at embedded line feed
"abc\ndef\n" =~ /^abc\n$/         # Doesn't match after embedded line feed
"abc\ndef\n" =~ /^abc\ndef$/      # Matches at string-ending line feed
"abc\ndef\n" =~ /^abc\ndef\n$/    # Matches at end of string

It's equivalent to \Z, which is equivalent to (?=\n\z|\z).

With /m

$ matches before a line feed, and at the end of the string.

"abc\ndef\n" =~ /^abc$/           # Matches at embedded line feed
"abc\ndef\n" =~ /^abc\n$/         # Doesn't match after embedded line feed
"abc\ndef\n" =~ /^abc\ndef$/      # Matches at string-ending line feed
"abc\ndef\n" =~ /^abc\ndef\n$/    # Matches at end of string

It's equivalent to (?=\n|\z).

---

\z is used when you want an exact match.

/xyz\z/    # String ends with "xyz"

$ is used when you want to ignore a trailing line feed.

/xyz$/     # Line ends with "xyz". The string might end with a line feed.

For example,

"jkl"   =~ /^jkl$/     # Matches at end of string
"jkl"   =~ /^jkl\z/    # Matches at end of string

"jkl\n" =~ /^jkl$/     # Matches at string-ending line feed
"jkl\n" =~ /^jkl\z/    # Doesn't match at string-ending line feed

$ is useful if matching against lines you haven't chomped yet.

while (<>) {
   next if /^foo$/;
   ...
}

\z is useful the rest of the time.

---

Note that other regex engines may behave differently, even those that are Perl-like. For example, in JavaScript, $ without /m only matches at the end of the string.

Answer 3

the point is that $ will match both before (a) newline character(s) and at the end of a file or an input string, which can or can not end with (a) newline character(s)

Answer 4

Please see if the following code any help in clarifying meaning of $ in regex, \n added for comparison

use strict;
use warnings;
use feature 'say';

my $str = 'abc
foo
bar
';

my $str_test;

$str_test = $str;
$str_test =~ s/(.+)$/[$1]/;
say '-' x 30;
say ' regex :: s/(.+)$/[$1]/';
say '-' x 30;
say $str_test;

$str_test = $str;
$str_test =~ s/(.+)$/[$1]/s;
say '-' x 30;
say ' regex :: s/(.+)$/[$1]/s';
say '-' x 30;
say $str_test;

$str_test = $str;
$str_test =~ s/\n/[NL]\n/s;
say '-' x 30;
say ' regex :: s/\n/[NL]\n/s';
say '-' x 30;
say $str_test;

$str_test = $str;
$str_test =~ s/\n/[NL]\n/g;
say '-' x 30;
say ' regex :: s/\n/[NL]\n/g';
say '-' x 30;
say $str_test;

Output

------------------------------
 regex :: s/(.+)$/[$1]/
------------------------------
abc
foo
[bar]

------------------------------
 regex :: s/(.+)$/[$1]/s
------------------------------
[abc
foo
bar
]
------------------------------
 regex :: s/\n/[NL]\n/s
------------------------------
abc[NL]
foo
bar

------------------------------
 regex :: s/\n/[NL]\n/g
------------------------------
abc[NL]
foo[NL]
bar[NL]