How can I match square bracket in regex with grep?

Question

I am trying to match both [ and ] with grep, but only succeeded to match [. No matter how I try, I can't seem to get it right to match ].

Here's a code sample:

echo "fdsl[]" | grep -o "[ a-z]\+" #this prints fdsl
echo "fdsl[]" | grep -o "[ \[a-z]\+" #this prints fdsl[
echo "fdsl[]" | grep -o "[ \]a-z]\+" #this prints nothing
echo "fdsl[]" | grep -o "[ \[\]a-z]\+" #this prints nothing

Edit: My original regex, on which I need to do this, is this one:

echo "fdsl[]" | grep -o "[ \[\]\t\na-zA-Z\/:\.0-9_~\"'+,;*\=()$\!@#&?-]\+" 
#this prints nothing

N.B: I have tried all the answers from this post but that didn't work on this particular case. And I need to use those brackets inside [].

Answer 1

According to BRE/ERE Bracketed Expression section of POSIX regex specification:

1. [...] The right-bracket ( ']' ) shall lose its special meaning and represent itself in a bracket expression if it occurs first in the list (after an initial circumflex ( '^' ), if any). Otherwise, it shall terminate the bracket expression, unless it appears in a collating symbol (such as "[.].]" ) or is the ending right-bracket for a collating symbol, equivalence class, or character class. The special characters '.', '*', '[', and '\' (period, asterisk, left-bracket, and backslash, respectively) shall lose their special meaning within a bracket expression.

and

7. [...] If a bracket expression specifies both '-' and ']', the ']' shall be placed first (after the '^', if any) and the '-' last within the bracket expression.

Therefore, your regex should be:

echo "fdsl[]" | grep -Eo "[][ a-z]+"

Note the E flag, which specifies to use ERE, which supports + quantifier. + quantifier is not supported in BRE (the default mode).

The solution in Mike Holt's answer "[][a-z ]\+" with escaped + works because it's run on GNU grep, which extends the grammar to support \+ to mean repeat once or more. It's actually undefined behavior according to POSIX standard (which means that the implementation can give meaningful behavior and document it, or throw a syntax error, or whatever).

If you are fine with the assumption that your code can only be run on GNU environment, then it's totally fine to use Mike Holt's answer. Using sed as example, you are stuck with BRE when you use POSIX sed (no flag to switch over to ERE), and it's cumbersome to write even simple regular expression with POSIX BRE, where the only defined quantifier is *.

Original regex

Note that grep consumes the input file line by line, then checks whether the line matches the regex. Therefore, even if you use P flag with your original regex, \n is always redundant, as the regex can't match across lines.

While it is possible to match horizontal tab without P flag, I think it is more natural to use P flag for this task.

Given this input:

$ echo -e "fds\tl[]kSAJD<>?,./:\";'{}|[]\\!@#$%^&*()_+-=~\`89"
fds     l[]kSAJD<>?,./:";'{}|[]\!@#$%^&*()_+-=~`89

The original regex in the question works with little modification (unescape + at the end):

$ echo -e "fds\tl[]kSAJD<>?,./:\";'{}|[]\\!@#$%^&*()_+-=~\`89" | grep -Po "[ \[\]\t\na-zA-Z\/:\.0-9_~\"'+,;*\=()$\!@#&?-]+"
fds     l[]kSAJD
?,./:";'
[]
!@#$
&*()_+-=~
89

Though we can remove \n (since it is redundant, as explained above), and a few other unnecessary escapes:

$ echo -e "fds\tl[]kSAJD<>?,./:\";'{}|[]\\!@#$%^&*()_+-=~\`89" | grep -Po "[ \[\]\ta-zA-Z/:.0-9_~\"'+,;*=()$\!@#&?-]+"
fds     l[]kSAJD
?,./:";'
[]
!@#$
&*()_+-=~
89

Answer 2

One issue is that [ is a special character in expression and it cannot get escaped with \ (at least not in my flavors of grep). Solution is to define it like [[].

Answer 3

According to regular-expressions.info:

In most regex flavors, the only special characters or metacharacters inside a character class are the closing bracket (]), the backslash (\), the caret (^), and the hyphen (-). The usual metacharacters are normal characters inside a character class, and do not need to be escaped by a backslash.

... and ...

The closing bracket (]), the caret (^) and the hyphen (-) can be included by escaping them with a backslash, or by placing them in a position where they do not take on their special meaning.

So, assuming that the particular flavor of regular expressions syntax supported by grep conforms to this, then I would have expected that "[ a-z[\]]\+" should have worked.

However, my version of grep (GNU grep 2.14) only matches the "[]" at the end of "fdsl[]" with this regex.

However, I tried using the other technique mentioned in that quote (putting the ] in a position within the character class where it cannot take on its normal meaning, and it seems to have worked:

$ echo "fdsl[]" | grep -o "[][a-z ]\+"
fdsl[]