2

I am a newbie in bash script.

Here is my environment:

Mac OS X Catalina

/bin/bash

I found here a mix of several commands to remove the duplicate string in a string.

I needed for my program which updates the .zhrc profile file.

Here is my code:

#!/bin/bash
a='export PATH="/Library/Frameworks/Python.framework/Versions/3.8/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/local/bin:"'

myvariable=$(echo "$a" | tr ':' '\n' | sort | uniq | xargs)

echo "myvariable : $myvariable"

Here is the output:

xargs: unterminated quote
myvariable :

After some test, I know that the source of the issue is due to some quotes "" inside my variable '$a'.

Why am I so sure?

Because when I execute this code for example:

#!/bin/bash
a="/Library/Java/JavaVirtualMachines/jdk1.8.0_271.jdk/Contents/Home:/Library/Java/JavaVirtualMachines/jdk1.8.0_271.jdk/Contents/Home"

myvariable=$(echo "$a" | tr ':' '\n' | sort | uniq | xargs)

echo "myvariable : $myvariable"

where $a doesn't contain any quotes, I get the correct output:

myvariable : /Library/Java/JavaVirtualMachines/jdk1.8.0_271.jdk/Contents/Home

I tried to search for a solution for "xargs: unterminated quote" but each answer found on the web is for a particular case which doesn't correspond to my problem.

As I am a newbie and this line command is using several complex commands, I was wondering if anyone know the magic trick to make it work.

Gauthier Buttez
  • 1,005
  • 1
  • 16
  • 40
  • Thank you so much. It almost works. I get 99% correct output. It misses the 2nd quote " at the end => export PATH="/Library/Frameworks/Python.framework/Versions/3.8/bin – Gauthier Buttez Dec 14 '20 at 15:42
  • The code as a whole is pretty dangerous: You split the string in `a` at the colons into several lines. Therefore one of the strings starts with `export` and ends with `Home`, and has a quote somewhere in the middle. Another string ends with a quote. Then you sort them, and then glue this pieces together. It is not clear that the parts will end up in a way that the result is a well-formed command again, where the quotes are in the right place. – user1934428 Dec 14 '20 at 15:48
  • A simplified version of the problem with your approach can be shown with `(echo '"a'; echo 'b"')|xargs`, which results in an _unmatched double quote_ error message. – user1934428 Dec 14 '20 at 15:52
  • you might consider reading https://stackoverflow.com/questions/18135451/what-is-the-difference-between-var-var-and-var-in-the-bash-shell – smed Dec 14 '20 at 16:28
  • 1
    one issue with `sort`ing is that you'll likely change the order of the entries which in turn could have undesired consequences (eg, `$PATH` was originally built with a specific precedence/directory-ordering in mind, but that precedence/ordering has been mangled due to the `sort`ing); net result is that for variables like `$PATH` you'll probably want to look at solutions that maintain the order of the individual directories – markp-fuso Dec 15 '20 at 14:40

2 Answers2

0

Basically, you want to remove duplicates from a colon-separated list.

I don't know if this is considered cheating, but I would do this in another language and invoke it from bash. First I would write a script for this purpose in zsh: It accepts as parameter a string with colon separtors and outputs a colon-separated list with duplicates removed:

#!/bin/zsh

original=${1?Parameter missing}  # Original string

# Auxiliary array, which is set up to act like a Set, i.e. without 
# duplicates
typeset -aU nodups_array

# Split the original strings on the colons and store the pieces
# into the array, thereby removing duplicates. The core idea for
# this is stolen from:
# https://stackoverflow.com/questions/2930238/split-string-with-zsh-as-in-python
nodups_array=("${(@s/:/)original}")

# Join the array back with colons and write the resulting string
# to stdout.
echo ${(j':')nodups_array}

If we call this script nodups_string, you can invoke it in your bash-setting as:

#!/bin/bash
a_path="/Library/Frameworks/Python.framework/Versions/3.8/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/local/bin:"
nodups_a_path=$(nodups_string "$a_path")
my_variable="export PATH=$nodups_a_path"
echo "myvariable : $myvariable"

The overall effect would be literally what you asked for. However, there is still an open problem I should point out: If one of the PATH components happens to contain a space, the resulting export statement can not validly be executed. This problem is also inherent into your original problem; you just didn't mention it. You could do something like

my_variable=export\ PATH='"'$nodups_a_path"'"'

to avoid this. Of course, I wonder why you take such an effort to generat a syntactically valid export command, instead of simply building the PATH by directly where it is needed.

Side note: If you would use zsh as your shell instead of bash, and only want to keep your PATH free of duplicates, a simple

typeset -iU path

would suffice, and zsh takes care of the rest.

user1934428
  • 19,864
  • 7
  • 42
  • 87
0

With awk:

awk -v RS=[:\"] 'NR > 1 { pth[$0]="" } END { for (i in pth) { if (i !~ /[[:space:]]+/ && i != "" ) { printf "%s:",i } } }' <<< "$a"

Set the record separator to : and double quotes. Then when the number record is greater than one, set up an array called pth with the path as the index. At the end, loop through the array, re printing the paths separated with :

Raman Sailopal
  • 12,320
  • 2
  • 11
  • 18