I wouldn't call the observed behaviour a bug, because there is no specification for history expansion other than the observed behaviour of the bash shell itself. But it is certainly the case that the precise mechanics of parsing a history expansion expression is not well documented and has a lot of possibly surprising corner cases.
The bash manpage does state that history expansion "is performed immediately after a complete line is read, before the shell breaks it into words" (emphasis added), while the bash manual mentions that history expansion is provided by the History library. This is the root cause of most of the history expansion parsing oddities: history expansion works on raw unparsed input without any assistance from the bash tokenizer, and is mostly done with an external library which is not bash-specific. Since tokenizing bash input is non-trivial, it is not really surprising that the relatively simple parsing rules used during history expansion are only a rough approximation to a real bash parse.
For example, the bash manual does indicated that you can prevent a history expansion character (!) from being recognized as such by backslash-quoting it. But it is not explicitly documented that any \ which immediately precedes an ! will inhibit recognition of the history expansion, even if the backslash was itself quoted with a backslash. So the ! in \\!word does not cause the previous command starting with word to be substituted. (\\word is a common way to execute the command word instead of the alias word, so the example is not entirely contrived.)
A longer discussion of some of the corner cases of the recognition of the history expansion character can be found in this answer.
The issue raised by this question is slightly different, since it is about the next phase of the history expansion parse. Once it has been established that a particular character is a history expansion character, it is then necessary to parse the "event" which follows; as indicated by the bash manual, the event can take several forms, one of which is !string, representing the most recent command which starts with "string".
It is implied that this form will only be used if no other form applies, which means that string may not start with a digit or -, !, # or ?. It also may not start with whitespace or = (since those would inhibit history expansion) and in some circumstances ( or " (which may inhibit history expansion). And finally, it may not start with ^, $, % or *, which would be interpreted as a word designator (from the default event, which is the previous command).
The bash manual does not specify what terminates the string. It is semi-documented in the history library manual, which mentions that a history search string (or "event" as it is called in the bash manual) is terminated by whitespace, :, or any of the characters in the history configuration variable history_search_delimiter_chars. (For the record, bash currently (v4.3) sets that variable to ";&()|<>".)
As indicated earlier, quoting is taken into account when deciding whether or not to recognize a history expansion character; as it turns out, if the history expansion occurs inside a double-quoted string, then the closing double-quote is also considered a history search delimiter character. And that, as far as I know, is the entire list of characters which will delimit !string.
Nowhere in either the bash nor the history documentation does it state that a history search delimiter character can be made non-special by quoting, and indeed this does not happen. An open quote, whether double or single, or even a backslash following the ! will be treated as just part of the string to be searched for, without any special processing.
Parsing of the substring-match history expansion -- !?string? -- is completely different. That string can only be terminated by a ? or by a newline. (As the bash manual says, the trailing ? is optional if terminated by a newline.)
Once the history expansion character has been recognized and the history search string has been identified, it may then be necessary to split the retrieved history entry into words. Again, the bash manual is slightly cavalier about corner cases, when it says that "the line is broken into words in the same fashion that Bash does, so that several words surrounded by quotes are considered one word."
A pedant would observe that "in the same fashion that Bash does" is not quite the same as saying "exactly as Bash would do", and in fact the second part of the sentence is literall true: several words surrounded by quotes are considered one word even if the quotes are not really matching quotes. For example, the line:
command "$(echo " foo bar ")"
is considered by the history library to consist of the following five words:
0. command
1. "$(echo "
2. foo
3. bar
4. ")"
although the bash parse would be quite different. By contrast, bash and the history library agree on the parsing of
command "$(echo ' foo bar ')"
as two words.
