8

I am on GNU bash, version 4.3.11.

Say I want to print unique lines on a file. I am using this approach, which works well on a file:

$ cat a
9
10
9
11
$ awk '!seen[$0]++' a
9
10
11

However, if I get the input from stdin, using double quotes in a multi-line and piping to awk, it fails:

$ echo "9
> 10
> 9
> 11" | awk '!seen[$0]++'
bash: !seen[$0]++': event not found

That is, bash tries to expand the command seen, which of course does not know because it is a variable name. But it shouldn't happen, since the command is placed within single-quotes.

echoing in a single-quoted, multi-line input works well:

$ echo '9
> 10
> 9
> 11' | awk '!seen[$0]++'
9
10
11

The funny thing is that it also works well on a single-line input that is double-quoted:

$ printf "9\n10\n9\n11" | awk '!seen[$0]++'
9
10
11

I wonder why is Bash trying to expand history if it occurs after a mutiline input, even though the command itself uses single-quotes.

Other considerations:

Having a pipe in between does not fix it, either:

$ echo "9
> 10
> 9
> 11" | cat - | awk '!seen[$0]++'
bash: !seen[$0]++': event not found

And setting set +H turns history off, so it works well because it does not try to expand anything:

$ set +H
$ echo "9
> 10
> 9
> 11" | awk '!seen[$0]++'
9
10
11

I went through the canonical answer by rici on how to address error “bash: !d': event not found” in Bash command substitution and found many possible reasons, but none matches this behaviour.

Community
  • 1
  • 1
fedorqui
  • 275,237
  • 103
  • 548
  • 598
  • 1
    It appears that the last line is parsed as a single line, instead of include the before quoted string. Meaning that the `!` is processed as though it is in double quotes. You will also notice that you will get a `>` on the next line as it waits for you to close this perceived quoted string. This is why it works on a single line as well. Looks like a bug. – 123 Jun 14 '16 at 09:48
  • 1
    Reproducible in 4.2.47. I'd file a bug. – choroba Jun 14 '16 at 09:49
  • Also reproducible all the way back to `3.2.51` – 123 Jun 14 '16 at 09:53
  • 2
    `$ ls` `$ echo "asdasd" | echo '!!'` will print weird outputs... – anishsane Jun 14 '16 at 09:54
  • 1
    @anishsane same with `!#`, not what you would expect, try `echo "123" | awk '!#'`. It returns `" | awk '" | awk ''` – 123 Jun 14 '16 at 10:06
  • Also note that, this issue is reproducible only if line containing the second command starts with `"`. If you add a multiline single quoted string in between, it works correctly. – anishsane Jun 14 '16 at 11:18
  • @anishsane yep, thanks! I mentioned this in the question. – fedorqui Jun 14 '16 at 11:19
  • Need your opinions on http://stackoverflow.com/questions/37816059/bash-hashmap-using-quote-as-key too... – anishsane Jun 14 '16 at 15:43
  • @anishsane interesting one! I see the VIP in bash already handled it : ) – fedorqui Jun 15 '16 at 06:39
  • Check [this](http://stackoverflow.com/questions/6697753/difference-between-single-and-double-quotes-in-bash) out! – meain Jun 18 '16 at 17:43
  • @meain why is this relevant? – fedorqui Jun 19 '16 at 13:45
  • @choroba so I thought. I filed a bug and got an answer stating that this is kind of "normal". – fedorqui Jun 20 '16 at 08:13

1 Answers1

2

It is not a bug.

After asking this in the bash-bugs mailing list, I got the following answer:

History expansion is explicitly line-oriented.

It doesn't know about shell state, especially shell quoting state, that spans lines.

It does know about vaguely shell-like quoting that's common across a large set of Unix utilities -- since the history and readline libraries are used outside the shell -- and that a double quote introduces a quoted string in which single quotes are not significant and don't inhibit history expansion.

fedorqui
  • 275,237
  • 103
  • 548
  • 598