"sed" doesn't match pattern

Question

I'm trying to format cut, paste output but sed not working.

file.txt

Apple
Banana
Apple
Banana
Orange
Apple
Orange

code.sh

cut -f2 file.txt | sort | uniq | sed 's/^\|$/#/g'| paste -sd,\& -

expected output / output on ubuntu

#Apple#,#Banana#&#Orange#

getting output / output on macos

Apple,Banana&Orange

Note: The code works on Ubuntu, but on MacOS it doesn't.

macos sed probably doesn't support `\|`.. you can try `sed -E 's/^|$/#/g'` see also: https://stackoverflow.com/questions/24275070/sed-not-giving-me-correct-substitute-operation-for-newline-with-mac-difference/24276470#24276470 — Sundeep, Feb 20 '21 at 06:23
This problem has nothing to do with `cut`, `sort`, `uniq`, and `paste`. You could reduce the whole example to `echo foo | sed 's/^\|$/#/g'` — Barmar, Feb 20 '21 at 06:25
@Sundeep Weirdly, that inserts `#` at the beginning of each line, but not the end. — Barmar, Feb 20 '21 at 06:26
@Barmar weird indeed, I can't think of a reason why it wouldn't work — Sundeep, Feb 20 '21 at 06:27
@Barmar Yes `sort -u` is working on Mac but does it work on Ubuntu? — , Feb 20 '21 at 06:45

score 2 · Answer 1 · answered Feb 20 '21 at 08:20

2

This can be done in a single gnu-awk:

awk '!seen[$1]++{} END {
    PROCINFO["sorted_in"]="@ind_str_asc"
    for (i in seen)
      s = s (s == "" ? "" : (++j==1?",":"&")) "#" i "#"
    print s
}' file

#Apple#,#Banana#&#Orange#

On OSX I have gnu awk installed via home brew.

answered Feb 20 '21 at 08:20

anubhava

761,203
64
569
643

Looks good, I have home brew, but I don't use awk. – Feb 24 '21 at 14:34
It is super easy to install then `brew install gawk` – anubhava Feb 24 '21 at 14:35

Sundeep · Answer 2 · 2021-02-21T03:49:19.537

1

As far as I know, BSD/Mac sed doesn't support \|. See sed not giving me correct substitute operation for newline with Mac - differences between GNU sed and BSD / OSX sed for details.

As an alternate, you can use ERE instead of BRE. I checked it on Linux, apparently this still doesn't seem to work on Mac (See also: MacOS sed: match either beginning or end).

$ echo 'Apple' | sed -E 's/^|$/#/g'
#Apple#

# workaround for Mac
$ echo 'Apple' | sed -e 's/^/#/' -e 's/$/#/'
#Apple#

Instead of sort+uniq+sed, you can also use awk (but note that awk solution shown here removes duplicates while preserving original order, doesn't sort the input):

$ awk '!seen[$0]++{print "#" $0 "#"}' ip.txt
#Apple#
#Banana#
#Orange#

Change $0 to $2 if you want only the second field, based on your use of cut

edited Feb 21 '21 at 03:49

answered Feb 20 '21 at 06:30

Sundeep

23,246
2
28
103

Maybe it depends on the OS version, but on 10.13.6 the `sed` command produces just `#Apple` – Barmar Feb 20 '21 at 07:15
@Barmar I have only Linux, so can't test it elsewhere. Edited the answer to reflect that. – Sundeep Feb 20 '21 at 07:33
2

Sundeep and @Barmar - see https://superuser.com/a/1420167, you're hitting the `s/^|$/#/g` bug in sed on MacOS. – Ed Morton Feb 20 '21 at 17:54
Good find @EdMorton – Barmar Feb 20 '21 at 18:05
@Barmar, thanks. I'm sure there's other posts out there that provide C&V as I I've come across this before, I was just too lazy to look for them! – Ed Morton Feb 20 '21 at 18:12

score 1 · Accepted Answer · answered Feb 20 '21 at 06:34

1

As mentioned elsewhere, BSD sed doesn't support \|. Instead of replacing ^ and $, you can substitute # around the whole line.

sort -u file.txt | sed 's/.*/#&#/' | paste -sd,'&' -

answered Feb 20 '21 at 06:34

Barmar

741,623
53
500
612

Thanks for `sort -u` but does it work with ubuntu? – Feb 20 '21 at 06:47
1

It should work everywhere, it's a standard `sort` option that long predates all the different flavors of Unix. – Barmar Feb 20 '21 at 06:50
I realised that your solution is properly working, but tell me one thing why you replaced `$` with `&` in `s/.*/#/`. With `$` it doesn't work. – Feb 20 '21 at 07:08
`&` is replaced with whatever the regexp matched, which is the whole line. – Barmar Feb 20 '21 at 07:14
`$` only has meaning in the regexp, not the replacement. – Barmar Feb 20 '21 at 07:16
1

Right, `-u` is a required argument for POSIX `sort`, see https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sort.html, and has been present in every sort I remember using for the past 40+ years. – Ed Morton Feb 20 '21 at 18:07

score 0 · Answer 4 · answered Oct 08 '21 at 15:32

A simple way to do it using the sed command:

sed -E 's/[[:alnum:]]+/#&#/'

the -E option for enabling the POSIX ERE (extended regular
expression)
[[:alnum:]]+ The alphanumeric characters; in ASCII, equivalent to [A-Za-z0-9] with the plus (+) to refer to one or more.
the & symbol, does bring or refer to the content of the pattern we found. (on which we surrounded it with #)

"sed" doesn't match pattern

4 Answers4