-2

I am looking for four simple, bullet-proof, as short as possible Bash functions:

PrependPath name path [path ...]
AppendPath name path [path ...]
RemovePath name path [path ...]
UniquePath name

that respectively:

  • Prepend path(s) to the Bash variable specified by name, if those path(s) are not already contained somewhere in that variable,
  • Append path(s) to the Bash variable specified by name, if those path(s) are not already contained somewhere in that variable,
  • Remove all instances of the given path(s) from the Bash variable specified by name,
  • Filter out all but the first (leftmost) occurrence of any duplicates in the variable.

For example (I have not forgotten any $ signs):

export PATH=/bin:/usr/bin:/bin
PrependPath PATH /usr/local/bin  # --> PATH=/usr/local/bin:/bin:/usr/bin:/bin
AppendPath PATH /usr/local/bin   # --> PATH=/usr/local/bin:/bin:/usr/bin:/bin  (nothing changed, as was already there from PrependPath)
RemovePath PATH /usr/local/bin   # --> PATH=/bin:/usr/bin:/bin
UniquePath PATH                  # --> PATH=/bin:/usr/bin

Once we have these functions, we can use them for any path variables following the same (or very similar rules) to the PATH variable, like for example LD_LIBRARY_PATH, LIBRARY_PATH, INFOPATH, ...

What makes it hard is that I want these functions to work for absolutely every legal path that can feasibly occur in PATH, which basically eliminates the characters \0, / and :, but allows all other characters (assuming it is validly encoded UTF-8), including in particular spaces and newlines.

I would like to keep the philosophical discussion of "no-one in their right mind would ever use a path like that" out of this. If I could legally create a path like that, and it's legal to put it in PATH, then it's fair game.

Some more notes:

  • The function should not modify the parent environment in any way (e.g. add functions/variables to it) other than exporting/changing the specified path variable.
  • The function should not equate paths like /usr/ and /usr, that differ just by slashes.
  • The function should deal correctly with empty paths (specifying the current directory), by not accidentally adding them to the path variable if they're not there, and not accidentally removing them if they are already there.
  • The function should work even if PATH is empty. That situation cannot be excluded as being impossible.

I have created a unit test script that embodies what I believe to be entirely reasonable and common-sense expectations of the behaviour of the four functions (I wanted to avoid an external link):

function PrependPath() { TODO }
function AppendPath() { TODO }
function RemovePath() { TODO }
function UniquePath() { TODO }

function Assert() { ((i++)); [[ "$TESTPATH" == "$1" ]] && echo "$i) SUCCESS" || echo "$i) FAILURE => Got >$TESTPATH<"; }

i=0
echo "Test: PrependPath"

TESTPATH="/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
PrependPath TESTPATH /usr/sbin
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
PrependPath TESTPATH /bin
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
PrependPath TESTPATH /usr/slash
Assert "/usr/slash:/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
PrependPath TESTPATH /new
Assert "/new:/usr/slash:/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
PrependPath TESTPATH /new
Assert "/new:/usr/slash:/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
PrependPath TESTPATH ""
Assert ":/new:/usr/slash:/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
PrependPath TESTPATH ""
Assert ":/new:/usr/slash:/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
PrependPath TESTPATH "/usr"
Assert "/usr::/new:/usr/slash:/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"

TESTPATH=
PrependPath TESTPATH /multiple /new
Assert "/new:/multiple"
PrependPath TESTPATH /and /new /foo /and
Assert "/foo:/and:/new:/multiple"

TESTPATH=":/foo:/bar:"
PrependPath TESTPATH /bar
Assert ":/foo:/bar:"

TESTPATH="/foo:/bar:"
PrependPath TESTPATH ""
Assert "/foo:/bar:"

TESTPATH=":/foo:/bar"
PrependPath TESTPATH ""
Assert ":/foo:/bar"

TESTPATH="/foo::/bar"
PrependPath TESTPATH ""
Assert "/foo::/bar"

TESTPATH=$'/te st\nnew/foo:/and:/ano\nth er:/more:/te st\nnew/foo:'
PrependPath TESTPATH /ano
Assert $'/ano:/te st\nnew/foo:/and:/ano\nth er:/more:/te st\nnew/foo:'
PrependPath TESTPATH $'/te st\nnew/foo'
Assert $'/ano:/te st\nnew/foo:/and:/ano\nth er:/more:/te st\nnew/foo:'
PrependPath TESTPATH $'/ne w\ner\n'
Assert $'/ne w\ner\n:/ano:/te st\nnew/foo:/and:/ano\nth er:/more:/te st\nnew/foo:'

TESTPATH="/foo:/bar"
PrependPath TESTPATH "/b.r" "/fo+" "/fo*" "/b[a]r" "\foo" "/food?"
Assert "/food?:\foo:/b[a]r:/fo*:/fo+:/b.r:/foo:/bar"

TESTPATH=$':/te[ st\nnew/foo:/a[d:/ano\nth er:/mor(e:'
PrependPath TESTPATH "/a[d" "/mor(e" "/other{" "/ano"
Assert $'/ano:/other{::/te[ st\nnew/foo:/a[d:/ano\nth er:/mor(e:'

TESTPATH=$'\new'
PrependPath TESTPATH "\new"
Assert $'\\new:\new'

TESTPATH=
PrependPath TESTPATH "/foo"
Assert "/foo"
PrependPath TESTPATH ""
Assert ":/foo"

TESTPATH=$'\n:/foo:/bar:\n'
PrependPath TESTPATH "/bar"
Assert $'\n:/foo:/bar:\n'
PrependPath TESTPATH $'\n'
Assert $'\n:/foo:/bar:\n'
PrependPath TESTPATH "/new"
Assert $'/new:\n:/foo:/bar:\n'
PrependPath TESTPATH ""
Assert $':/new:\n:/foo:/bar:\n'

TESTPATH=$':/foo:/bar:\n\n'
PrependPath TESTPATH "/bar"
Assert $':/foo:/bar:\n\n'
PrependPath TESTPATH $'\n'
Assert $'\n::/foo:/bar:\n\n'
PrependPath TESTPATH $'\n\n'
Assert $'\n::/foo:/bar:\n\n'
PrependPath TESTPATH "/new"
Assert $'/new:\n::/foo:/bar:\n\n'

TESTPATH=
PrependPath TESTPATH ""
Assert ":"
PrependPath TESTPATH ""
Assert ":"
PrependPath TESTPATH /new
Assert "/new:"
PrependPath TESTPATH ""
Assert "/new:"

TESTPATH=":/bin:"
PrependPath TESTPATH ""
Assert ":/bin:"
PrependPath TESTPATH "/foo"
Assert "/foo::/bin:"

TESTPATH="::"
PrependPath TESTPATH ""
Assert "::"
PrependPath TESTPATH "/foo"
Assert "/foo:::"

TESTPATH=":::"
PrependPath TESTPATH ""
Assert ":::"
PrependPath TESTPATH "/foo"
Assert "/foo::::"

ORIGPATH="$PATH"
PATH=
TESTPATH="/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
PrependPath TESTPATH /usr/sbin /bin /usr/slash /new /foo /new
Assert "/foo:/new:/usr/slash:/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
PATH="$ORIGPATH"

i=0
echo
echo "Test: AppendPath"

TESTPATH="/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
AppendPath TESTPATH /usr/sbin
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
AppendPath TESTPATH /bin
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
AppendPath TESTPATH /usr/slash
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/:/usr/slash"
AppendPath TESTPATH /new
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/:/usr/slash:/new"
AppendPath TESTPATH /new
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/:/usr/slash:/new"
AppendPath TESTPATH ""
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/:/usr/slash:/new:"
AppendPath TESTPATH ""
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/:/usr/slash:/new:"
AppendPath TESTPATH "/usr"
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/:/usr/slash:/new::/usr"

TESTPATH=
AppendPath TESTPATH /multiple /new
Assert "/multiple:/new"
AppendPath TESTPATH /and /new /foo /and
Assert "/multiple:/new:/and:/foo"

TESTPATH=":/foo:/bar:"
AppendPath TESTPATH /bar
Assert ":/foo:/bar:"

TESTPATH="/foo:/bar:"
AppendPath TESTPATH ""
Assert "/foo:/bar:"

TESTPATH=":/foo:/bar"
AppendPath TESTPATH ""
Assert ":/foo:/bar"

TESTPATH="/foo::/bar"
AppendPath TESTPATH ""
Assert "/foo::/bar"

TESTPATH=$'/te st\nnew/foo:/and:/ano\nth er:/more:/te st\nnew/foo:'
AppendPath TESTPATH /ano
Assert $'/te st\nnew/foo:/and:/ano\nth er:/more:/te st\nnew/foo::/ano'
AppendPath TESTPATH $'/te st\nnew/foo'
Assert $'/te st\nnew/foo:/and:/ano\nth er:/more:/te st\nnew/foo::/ano'
AppendPath TESTPATH $'/ne w\ner\n'
Assert $'/te st\nnew/foo:/and:/ano\nth er:/more:/te st\nnew/foo::/ano:/ne w\ner\n'

TESTPATH="/foo:/bar"
AppendPath TESTPATH "/b.r" "/fo+" "/fo*" "/b[a]r" "/bar" "\foo" "/food?"
Assert "/foo:/bar:/b.r:/fo+:/fo*:/b[a]r:\foo:/food?"

TESTPATH=$':/te[ st\nnew/foo:/a[d:/ano\nth er:/mor(e:'
AppendPath TESTPATH "/a[d" "/mor(e" "/other{" "/ano"
Assert $':/te[ st\nnew/foo:/a[d:/ano\nth er:/mor(e::/other{:/ano'

TESTPATH=$'\new'
AppendPath TESTPATH "\new"
Assert $'\new:\\new'

TESTPATH=
AppendPath TESTPATH "/foo"
Assert "/foo"
AppendPath TESTPATH ""
Assert "/foo:"

TESTPATH=$'\n:/foo:/bar:\n'
AppendPath TESTPATH "/bar"
Assert $'\n:/foo:/bar:\n'
AppendPath TESTPATH $'\n'
Assert $'\n:/foo:/bar:\n'
AppendPath TESTPATH "/new"
Assert $'\n:/foo:/bar:\n:/new'
AppendPath TESTPATH ""
Assert $'\n:/foo:/bar:\n:/new:'

TESTPATH=$':/foo:/bar:\n\n'
AppendPath TESTPATH "/bar"
Assert $':/foo:/bar:\n\n'
AppendPath TESTPATH $'\n'
Assert $':/foo:/bar:\n\n:\n'
AppendPath TESTPATH $'\n\n'
Assert $':/foo:/bar:\n\n:\n'
AppendPath TESTPATH "/new"
Assert $':/foo:/bar:\n\n:\n:/new'

TESTPATH=
AppendPath TESTPATH ""
Assert ":"
AppendPath TESTPATH ""
Assert ":"
AppendPath TESTPATH /new
Assert ":/new"
AppendPath TESTPATH ""
Assert ":/new"

TESTPATH=":/bin:"
AppendPath TESTPATH ""
Assert ":/bin:"
AppendPath TESTPATH "/foo"
Assert ":/bin::/foo"

TESTPATH="::"
AppendPath TESTPATH ""
Assert "::"
AppendPath TESTPATH "/foo"
Assert ":::/foo"

TESTPATH=":::"
AppendPath TESTPATH ""
Assert ":::"
AppendPath TESTPATH "/foo"
Assert "::::/foo"

ORIGPATH="$PATH"
PATH=
TESTPATH="/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/"
AppendPath TESTPATH /usr/sbin /bin /usr/slash /new /foo /new
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/slash/:/usr/slash:/new:/foo"
PATH="$ORIGPATH"

i=0
echo
echo "Test: RemovePath"

TESTPATH="/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/sbin/"
RemovePath TESTPATH /usr/sbin
Assert "/usr/bin:/bin:/usr/sbin/tmp:/usr/sbin/"

TESTPATH="/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/sbin/:/foo"
RemovePath TESTPATH /usr/bin /usr/sbin /foo
Assert "/bin:/usr/sbin/tmp:/usr/sbin/"

TESTPATH=$':/te st\nnew/foo:/and:/ano\nth er:/more:'
RemovePath TESTPATH /and
Assert $':/te st\nnew/foo:/ano\nth er:/more:'

TESTPATH=$':/te st\nnew/foo:/and:/ano\nth er:/more:/te st\nnew/foo:'
RemovePath TESTPATH $'/te st\nnew/foo'
Assert $':/and:/ano\nth er:/more:'

TESTPATH=$':/te st\nnew/foo:/and:/ano\nth er:/more:'
RemovePath TESTPATH ""
Assert $'/te st\nnew/foo:/and:/ano\nth er:/more'

TESTPATH=$':/te st\nnew/foo:/and:/ano\nth er:/more:'
RemovePath TESTPATH "/a.d" "\and" "/andy?" $'/te st\nnew/fo+' "/an*"
Assert $':/te st\nnew/foo:/and:/ano\nth er:/more:'

TESTPATH=$':/te[ st\nnew/foo:/a[d:/ano\nth er:/mor(e:'
RemovePath TESTPATH "/a[d" "/mor(e" "/other{" "/ano"
Assert $':/te[ st\nnew/foo:/ano\nth er:'

TESTPATH=$':/te st\nnew/foo:/and:/ano\nth er:\n:/more:'
RemovePath TESTPATH "/sp ace" $'/new\nline' $'\n' "" $'/ano\nth er'
Assert $'/te st\nnew/foo:/and:/more'

TESTPATH="\no:\newline"
RemovePath TESTPATH $'\no'
Assert "\no:\newline"
RemovePath TESTPATH "\no"
Assert "\newline"

TESTPATH=$'\n:/more:\n\n:/of:\n:/th is:\n\n:\n'
RemovePath TESTPATH ""
Assert $'\n:/more:\n\n:/of:\n:/th is:\n\n:\n'
RemovePath TESTPATH "\n"
Assert $'\n:/more:\n\n:/of:\n:/th is:\n\n:\n'
RemovePath TESTPATH $'\n\n'
Assert $'\n:/more:/of:\n:/th is:\n'
RemovePath TESTPATH $'\n'
Assert $'/more:/of:/th is'

TESTPATH=$'\n:/more:\n\n:/of:\n:/th is:\n\n:\n'
RemovePath TESTPATH $'\n'
Assert $'/more:\n\n:/of:/th is:\n\n'
RemovePath TESTPATH /of
Assert $'/more:\n\n:/th is:\n\n'

TESTPATH=$'\n:/more:\n\n:/of:\n:/th is::\n'
RemovePath TESTPATH $'\n'
Assert $'/more:\n\n:/of:/th is:'

TESTPATH=$'\n:/more:\n\n:/of:\n:/th is::\n'
RemovePath TESTPATH ""
Assert $'\n:/more:\n\n:/of:\n:/th is:\n'

TESTPATH=":::"
RemovePath TESTPATH "/foo"
Assert ":::"
RemovePath TESTPATH ""
Assert ""

TESTPATH="::"
RemovePath TESTPATH "/foo"
Assert "::"
RemovePath TESTPATH ""
Assert ""

TESTPATH=":"
RemovePath TESTPATH "/foo"
Assert ":"
RemovePath TESTPATH ""
Assert ""

TESTPATH="::/foo"
RemovePath TESTPATH "/foo"
Assert ":"

TESTPATH="::/foo"
RemovePath TESTPATH ""
Assert "/foo"

TESTPATH=":/foo"
RemovePath TESTPATH "/foo"
Assert ":"

TESTPATH=":/foo"
RemovePath TESTPATH ""
Assert "/foo"
RemovePath TESTPATH "/foo"
Assert ""

TESTPATH="/foo::"
RemovePath TESTPATH "/foo"
Assert ":"

TESTPATH="/foo::"
RemovePath TESTPATH ""
Assert "/foo"

TESTPATH="/foo:"
RemovePath TESTPATH "/foo"
Assert ":"

TESTPATH="/foo:"
RemovePath TESTPATH ""
Assert "/foo"

TESTPATH=""
RemovePath TESTPATH ""
Assert ""
RemovePath TESTPATH "/foo"
Assert ""

ORIGPATH="$PATH"
PATH=
TESTPATH="/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/sbin/"
RemovePath TESTPATH /usr/sbin /bin /foo
Assert "/usr/bin:/usr/sbin/tmp:/usr/sbin/"
PATH="$ORIGPATH"

i=0
echo
echo "Test: UniquePath"

TESTPATH="/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/sbin/"
UniquePath TESTPATH
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin/tmp:/usr/sbin/"
UniquePath TESTPATH
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin/tmp:/usr/sbin/"

TESTPATH="/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/sbin/:/usr/sbin:/bin"
UniquePath TESTPATH
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin/tmp:/usr/sbin/"

TESTPATH=":/foo:/bar:/foo"
UniquePath TESTPATH
Assert ":/foo:/bar"

TESTPATH="/foo:/bar:/foo:"
UniquePath TESTPATH
Assert "/foo:/bar:"

TESTPATH=":/foo:/bar:/foo:"
UniquePath TESTPATH
Assert ":/foo:/bar"

TESTPATH="/foo:/bar::/foo"
UniquePath TESTPATH
Assert "/foo:/bar:"

TESTPATH="/foo:/bar::/foo:"
UniquePath TESTPATH
Assert "/foo:/bar:"

TESTPATH=$':/te st\nnew/foo:/ano:/ano\nth er:th er::/more:/te st\nnew/foo:'
UniquePath TESTPATH
Assert $':/te st\nnew/foo:/ano:/ano\nth er:th er:/more'

TESTPATH=$'/te st\nnew/foo:/and:/ano\nth er:/more:/a.d:\\and:/andy?:/te st\nnew/fo+:/an*:/more'
UniquePath TESTPATH
Assert $'/te st\nnew/foo:/and:/ano\nth er:/more:/a.d:\\and:/andy?:/te st\nnew/fo+:/an*'

TESTPATH=$':/te[ st\nnew/foo:/a[d:/ano\nth er:/mor(e:/mor(e:/other{:/ano:/a[d'
UniquePath TESTPATH
Assert $':/te[ st\nnew/foo:/a[d:/ano\nth er:/mor(e:/other{:/ano'

TESTPATH=$'\\no:\newline:\no:\\newline:\no'
UniquePath TESTPATH
Assert $'\\no:\newline:\no:\\newline'

TESTPATH="/foo:/foo"
UniquePath TESTPATH
Assert "/foo"
UniquePath TESTPATH
Assert "/foo"

TESTPATH=":"
UniquePath TESTPATH
Assert ":"

TESTPATH="::"
UniquePath TESTPATH
Assert ":"

TESTPATH=":::"
UniquePath TESTPATH
Assert ":"

TESTPATH=$'\n:/more::/of:\n:/th is:\n\n:\n'
UniquePath TESTPATH
Assert $'\n:/more::/of:/th is:\n\n'

TESTPATH=$':/more:/of::\n\n::/th is:\n\n:\n'
UniquePath TESTPATH
Assert $':/more:/of:\n\n:/th is:\n'

TESTPATH=$'/foo:\n:/foo'
UniquePath TESTPATH
Assert $'/foo:\n'

TESTPATH=$'/foo::'
UniquePath TESTPATH
Assert $'/foo:'

TESTPATH=
UniquePath TESTPATH
Assert ""

ORIGPATH="$PATH"
PATH=
TESTPATH="/usr/bin:/usr/sbin:/bin:/usr/sbin:/usr/sbin/tmp:/usr/sbin/:/bin"
UniquePath TESTPATH
Assert "/usr/bin:/usr/sbin:/bin:/usr/sbin/tmp:/usr/sbin/"
PATH="$ORIGPATH"

If your functions pass every one of these tests, and no-one else can find a flaw that I haven't checked for in the script, then you have completely answered my question.

Naturally, none of the answers that I have tried to any other related questions on Stack Overflow have given me a solution to this problem (1, 2, 3, 4, 5, 6, 7, 8, 9). Maybe I missed one, but seeing as I'm looking for all four functions on the level of generality I specified, that still wouldn't completely answer my question.

What I've tried

The specification of the variable to modify using a string is easily dealt with using indirect expansion, e.g. if $1 is PATH then ${!1} is $PATH. Setting/exporting a variable indirectly can be done using export "${1}"=XXX. In a final solution there may be more efficient/elegant ways of dealing with multiple inputs, but as a fallback solution you can always just use a for-loop over them, so the task can be reduced to simply requiring four functions that perform the required operations on the PATH variable with a single argument.

Not polluting the parent namespace is achievable using the local keyword for variables, but subfunctions would only at best be unset -f, which doesn't really satisfy the requirement of not possibly changing the parent environment. Appending to the PATH variable can be neatly done using alternative parameter expansion, i.e. something like PATH="${PATH:+${PATH}:}/new/path". Prepending is then something like PATH="/new/path${PATH:+:${PATH}}".

These expressions deal correctly with adding a colon separator or not almost all the time, and for the special cases where they don't a simple conditional can probably be added to fix it up as appropriate. With all these building blocks at my disposal, the crux of my problem is how to robustly split up the path in a bullet-proof way in order to be able to check if a certain path is already present. One of my attempts was just to search the string without extracting the components:

function AppendPath() { echo "$PATH" | grep -Eq "(^|:)$1($|:)" || export PATH="$1${PATH:+:${PATH}}"; }

but this failed because the contents of $1 is interpreted as a regex. Furthermore, the call to grep fails when PATH is empty, meaning that an absolute path like /bin/grep would be required, but that doesn't feel particularly portable. For RemovePath, I tried using null characters as delimiters and doing an inverted grep, e.g. something like:

function RemovePath() { __path="$(echo -n "$PATH" | tr ':' '\0' | grep -zxvF "$1" | tr '\0' ':')"; export PATH="${__path%:}"; }

I used -F on grep to ensure that it doesn't get treated as a regex, matched whole lines with -x, used null delimiters with -z, and inverted the matching with -v. Aside from the obvious problems with certain corner cases and the empty PATH case, grep -zF still divides search patterns at every newline, despite using the null delimiter, meaning that trying to remove $'/foo\nbar' will instead just remove all instances of /foo and bar. A sneaky problem with command substitution $() comes to the surface here too, whereby it truncates all trailing newlines. Try:

echo -n "$(echo -n $'Hey\nthere\n\n\n')"

I tried using IFS=: to break up PATH into an array, but amongst other things command substitution is where I get stuck again when trying to reassemble it with something like "$(IFS=:; echo -n "${pathparts[*]}")". Right now I'm looking into a bash builtin-only idea that is based on the read command and manual looping in order to avoid command substitution, grep, and empty PATH problems. I actually wanted to avoid polluting my question with lots of things that don't work, but it was requested, and I hope at least now it is believed that I've given this a serious go myself prior to posting.

halfer
  • 19,824
  • 17
  • 99
  • 186
pallgeuer
  • 1,216
  • 1
  • 7
  • 17
  • I need this functionality on many machines from many different user profiles that I manage, and thus need to put these functions in many bashrc's and in other necessarily _self-contained_ scripts that also can't assume what user they're running from etc. Thus I don't care about how many characters it is or anything like that. Anything short-ish is fine. I just want to exclude 1 page answers with 3 sub-functions etc etc... – pallgeuer Apr 13 '20 at 14:45
  • 1
    This is just a very long request for someone to write the four functions for you. And frankly, you have probably put more time into writing the test cases than you will ever save by using the four functions over manually updating the path as necessary. – chepner Apr 13 '20 at 15:21
  • 1
    You say you haven't forgotten any `$` signs, but that way of calling your functions may not be the best – you'd need namerefs, indirection, or even `eval` in your function. I think you might get a better response if you reduce the scope (for example, ask only about adding something to `PATH`), show your attempt and how it fails. – Benjamin W. Apr 13 '20 at 15:30
  • 1
    I suggest your spend some time writing your own func which can meet all your requirements. Personally I have a `path()` for this which supports insert, delete, unique, list. It also supports specifying the VAR to manipulate (default is `PATH`). –  Apr 13 '20 at 15:47
  • If `TESTPATH` is empty and you wish to add the current directory to it, i.e. add `''`, then the only correct output `TESTPATH` that respects that wish is `:`... – pallgeuer Apr 13 '20 at 15:58
  • Automated scripts cannot 'save time' by manually updating paths, and every single user of bash on every linux OS can benefit from the answer to this question (which is a _lot_ of people), so it's not like I'm asking a weird or specific question that only a small number of people would find immensely useful. This is supported by there being endless other questions on Stack Overflow asking for adding/removing entries from `PATH`, but alas no answers that I could find yet that actually work in all cases. I have tried a lot to write one my own and so far failed, hence why I am seeking help...? – pallgeuer Apr 13 '20 at 15:59
  • Good to know! But `''` is also legal in PATH and is also interpreted as adding the current directory (try it out), so unless I'm missing something, `:` would still be the most natural response to requesting `''` to be added to an empty PATH... Otherwise you would have to start equating `''` and `.` everywhere in every check. – pallgeuer Apr 13 '20 at 16:15
  • Okay, will you show us what you've tried so far? – oguz ismail Apr 13 '20 at 17:28

1 Answers1

0

Here is a solution that to the extent of my testing will always do the right thing in every situation in managing PATH-like variables:

# Example: PrependPath PATH /usr/local/cuda/bin
function PrependPath() {
    local __args __item
    for __args in "${@:2}"; do
        [[ -n "${!1}" ]] && while IFS= read -r -d ':' __item; do
            [[ "$__item" == "$__args" ]] && continue 2
        done <<< "${!1}:"
        ([[ -n "${!1#:}" ]] || [[ -z "$__args" ]]) && __args+=':'
        export "${1}"="$__args${!1}"
    done
}

# Example: AppendPath PATH /usr/local/cuda/bin
function AppendPath() {
    local __args __item
    for __args in "${@:2}"; do
        [[ -n "${!1}" ]] && while IFS= read -r -d ':' __item; do
            [[ "$__item" == "$__args" ]] && continue 2
        done <<< "${!1}:"
        ([[ -n "${!1#:}" ]] || [[ -z "$__args" ]]) && __args=":$__args"
        export "${1}"="${!1}$__args"
    done
}

# Example: RemovePath INFOPATH /usr/local/texlive/2019/texmf-dist/doc/info
function RemovePath() {
    local __item __args __path=
    [[ -n "${!1}" ]] && while IFS= read -r -d ':' __item; do
        for __args in "${@:2}"; do
            [[ "$__item" == "$__args" ]] && continue 2
        done
        __path="$__path:$__item"
    done <<< "${!1}:"
    [[ "$__path" != ":" ]] && __path="${__path#:}"
    export "${1}"="$__path"
}

# Example: UniquePath LD_LIBRARY_PATH
function UniquePath() {
    local __item __args __path= __seen=()
    [[ -n "${!1}" ]] && while IFS= read -r -d ':' __item; do
        for __args in "${__seen[@]}"; do
            [[ "$__item" == "$__args" ]] && continue 2
        done
        __seen+=("$__item")
        __path="$__path:$__item"
    done <<< "${!1}:"
    [[ "$__path" != ":" ]] && __path="${__path#:}"
    export "${1}"="$__path"
}

The solution to the crux of the problem was to use read with : as the delimiter, and only use bash loops to check/assemble the required output path, without command substitution, grep, awk, sed or any other external commands.

Rolled up into 'one-liners' so that they can easily be used in scripts, on the command line, and in the .bashrc gives:

# Functions to manage arbitrary PATH-like variables
function PrependPath() { local __args __item; for __args in "${@:2}"; do [[ -n "${!1}" ]] && while IFS= read -r -d ':' __item; do [[ "$__item" == "$__args" ]] && continue 2; done <<< "${!1}:"; ([[ -n "${!1#:}" ]] || [[ -z "$__args" ]]) && __args+=':'; export "${1}"="$__args${!1}"; done; }
function AppendPath() { local __args __item; for __args in "${@:2}"; do [[ -n "${!1}" ]] && while IFS= read -r -d ':' __item; do [[ "$__item" == "$__args" ]] && continue 2; done <<< "${!1}:"; ([[ -n "${!1#:}" ]] || [[ -z "$__args" ]]) && __args=":$__args"; export "${1}"="${!1}$__args"; done; }
function RemovePath() { local __item __args __path=; [[ -n "${!1}" ]] && while IFS= read -r -d ':' __item; do for __args in "${@:2}"; do [[ "$__item" == "$__args" ]] && continue 2; done; __path="$__path:$__item"; done <<< "${!1}:"; [[ "$__path" != ":" ]] && __path="${__path#:}"; export "${1}"="$__path"; }
function UniquePath() { local __item __args __path= __seen=(); [[ -n "${!1}" ]] && while IFS= read -r -d ':' __item; do for __args in "${__seen[@]}"; do [[ "$__item" == "$__args" ]] && continue 2; done; __seen+=("$__item"); __path="$__path:$__item"; done <<< "${!1}:"; [[ "$__path" != ":" ]] && __path="${__path#:}"; export "${1}"="$__path"; }
pallgeuer
  • 1,216
  • 1
  • 7
  • 17