0

I see this throughout shunit2, which aims for maximum portability across old Bourne-style shell scripts (see source here):

# Determine if `builtin` command exists.
__SHUNIT_BUILTIN='builtin'
# shellcheck disable=2039
if ! ("${__SHUNIT_BUILTIN}" echo 123 >/dev/null 2>&1); then
  __SHUNIT_BUILTIN=''
fi

# Some more code ...

# ...and now a check

if ${__SHUNIT_BUILTIN} [ ... ]; then

Isn't [ a special builtin in all Bourne-style shells? (i.e. ash /bin/bash /bin/dash /bin/ksh /bin/mksh /bin/pdksh /bin/zsh /usr/xpg4/bin/sh /bin/sh /sbin/sh)

If builtin exists in the shell, why would you ever need to write if builtin [ ... ]; then? It's literally before every single if statement in the script.

UPDATE:

@tripleee gave the correct answer below: [ could be aliased. For posterity, I'm putting the POSIX shell evaluation order here:

  1. Collect each line between pipes (called "pipelines") from standard input, which contain 1 or more commands

  2. Break the pipeline into commands

  3. Set up I/O for the pipeline

  4. For each command

    (a) Split the command into tokens

    (b) If first token is a keyword (without quotes or backslashes) and is an opening keyword (e.g. if, {, or (), then

    i. Set up internally for a compound command

    ii. Read the next command and continue until compound command closed

    (c) Check the first token of each command against the list of aliases and if found, substitute the alias and go back to step (a)

    (d) Perform tilde substitutions (i.e. replace ~ for $HOME, if ~ present)

    (e) Perform variable substitutions (e.g. $variable)

    (f) Perform command substitutions (i.e. $()'s)

    (g) Perform arithmetic substitutions (i.e. $(())'s)

    (h) Use $IFS to split the line resulting from steps (e) through (g) into more tokens

    (i) Perform file wildcard expansion (i.e. expand *,?, and [...] pairs).

    (j) Search tokens that are commands for the path to their binary using the search order special built-ins --> functions --> regular built-ins --> $PATH

    (k) Run command after setting up I/O redirection

I forgot that step (c) happens way before step (j).

adam.hendry
  • 4,458
  • 5
  • 24
  • 51
  • 1
    I don't believe `[` is a _special_ builtin -- I'm 70% sure it's just a regular built-in. – Charles Duffy May 28 '21 at 17:59
  • (Check the POSIX spec -- the special builtins all have a link to the "special built-in" definition in the SEE ALSO section; it's not there for `test`). – Charles Duffy May 28 '21 at 18:05
  • @CharlesDuffy On every single one of the Bourne shells I listed (at least in their most recent versions), running `type [` and `type test` yields `[ is a shell builtin` and `test is a shell builtin` and running `command -v [` and `command -v test` each return `[` and `test`, respectively, indicating they aren't in `/usr/bin/` as a regular builtin. Thus, as far as I can tell, they are special builtins. – adam.hendry May 28 '21 at 18:23
  • Presence in `/usr/bin` is really beside the point -- several operating systems have a `/usr/bin/exit`, a `/usr/bin/cd`, etc. despite such utilities being objectively useless except when implemented as special built-ins. Regular built-ins are still internal shell implementations; the built-in implementations are not executables in `/usr/bin`. – Charles Duffy May 28 '21 at 18:42
  • As for the demonstration of `command -v` and `type`, those demonstrate builtin-ness, but not _special_ builtin-ness. **Regular built-ins are still built into the shell itself**, though the operating system can also provide an external-to-the-shell version invoked as an executable alongside the shell-builtin one. – Charles Duffy May 28 '21 at 18:43
  • @CharlesDuffy You're right: https://www.ibm.com/docs/en/aix/7.2?topic=c-command-command. So...if a particular regular built-in is not associated with a PATH environment variable search, how could it get overwritten and where does it live? – adam.hendry May 28 '21 at 19:53
  • @CharlesDuffy Question 2: Does POSIX shell have a way to check for special vs regular builtins? – adam.hendry May 28 '21 at 19:55
  • A regular built-in can be overwritten by shadowing it with a function, as described in the specification you quote above. – Charles Duffy May 28 '21 at 20:23
  • I don't know that there's a formally specified way to check, though _attempting_ to shadow a builtin with a function and seeing which gets called is an easy way to distinguish. – Charles Duffy May 28 '21 at 20:25
  • @CharlesDuffy Oh okay. So like for `echo`, I'd have to do `echo() { echo 'hi'; }` and then check if `type echo` has the word `function` in it? If it doesn't, then it's a special builtin because it wasn't overwritten. – adam.hendry May 28 '21 at 20:29
  • Hmm. bash doesn't appear to implement the spec properly, if the quote above is correct -- `( cd() { :; }; type cd )` reports that `cd` is a function despite the spec you quoted indicating that special builtins take precedence over functions, and `cd` is _definitely_ specified to be a special builtin. – Charles Duffy May 28 '21 at 20:40
  • @CharlesDuffy That is weird...using your code with `type -a` shows me it is both a function and a built-in, but it should show it only as a builtin (i.e. it should not get overwritten). If I try to use `cd`, it does nothing, certainly meaning it was overwritten. Using `unset -f cd` restores its functionality. The same happens in `zsh`. – adam.hendry May 28 '21 at 20:53
  • zsh isn't _expected_ to be POSIX-flavored, so I don't expect it to try. Whereas bash has a fairly small set of high-profile deviations (`echo -e` being one of the worst), but mostly keeps its extensions in unspecified space. – Charles Duffy May 28 '21 at 21:23
  • ...where exactly in the POSIX spec is the quote about evaluation order edited into the question taken from? I don't find it anywhere in https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html – Charles Duffy May 28 '21 at 21:23
  • @CharlesDuffy I was paraprashing the Shell Introduction (section 2.1) and "Command Search and Execution" section (Section https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_01_01). It's after Section 2.9.1. – adam.hendry May 28 '21 at 21:33
  • @CharlesDuffy Wow, this is terrible! I can overwrite `builtin`, `unset`, and `type`. Someone could completely hose my system this way! – adam.hendry May 28 '21 at 21:33
  • Eh, someone who can set up arbitrary code to run on login can already hose your system -- there's no reason someone who wants to hose your system and has that level of access would leave you with an unmodified shell binary at all. – Charles Duffy May 28 '21 at 21:44
  • Ahh: When `set -o posix` is active, bash doesn't allow functions to be defined with the name of a special builtin. – Charles Duffy May 28 '21 at 21:45
  • @CharlesDuffy Good point. And I tried `set -o posix`, but it doesn't work for me. I'm on GNU bash 5.0.17 and I can still overwrite `cd` with a function. – adam.hendry May 28 '21 at 21:53
  • @CharlesDuffy Does it work for you? – adam.hendry May 28 '21 at 22:00
  • Yes, it does work for me; when `set -o posix` is done before the attempted declaration of a function named `cd`, that function declaration fails. – Charles Duffy May 28 '21 at 22:56
  • @CharlesDuffy Would you mind showing me an example here: https://stackoverflow.com/questions/67745982/set-o-posix-not-working-in-bash-5-0-17-on-wsl2-ubuntu-20-04. In interactive terminal, I've tried `set -o posix; cd() { :; }; cd ~`, which all "works" for me (I can overwrite `cd`). I don't get any failure messages. – adam.hendry May 28 '21 at 23:21
  • @CharlesDuffy Please see https://stackoverflow.com/questions/67745982/set-o-posix-not-working-in-bash-5-0-17-on-wsl2-ubuntu-20-04/67747512?noredirect=1#comment119747890_67747512. I'm curious how you got yours to work. – adam.hendry May 29 '21 at 04:20
  • @CharlesDuffy I have good reason to suspect you weren't able to actually get unchanging `cd` to work. Please see stackoverflow.com/questions/67745982/…. – adam.hendry May 29 '21 at 17:11
  • I have every intention of getting back to you on this -- please be patient. I'm doing some work for my wife's (non-software-development-related) business; the systems I use for aforementioned side job are Windows-y; I'd need to be back at one of my Linux hosts to repro. – Charles Duffy May 29 '21 at 17:43
  • @CharlesDuffy No problem. No need to get back. I just wanted to point you to the fact that `cd` is a _regular_ builtin, not a _special_ builtin per the POSIX standard, so setting `set -o posix` won't prevent overwritting its behavior. – adam.hendry May 29 '21 at 17:53
  • I wonder if bash changed behavior on this between 4.x and 5.x -- I tested against the former. – Charles Duffy May 29 '21 at 18:18
  • @CharlesDuffy And as always, THANK YOU SO MUCH for all your help! I greatly appreciate how quick your responses have been and how much you have helped me understand things! – adam.hendry May 29 '21 at 18:18
  • @CharlesDuffy Oh that would be frustrating! I asked the person who posted the answer in the question to check against 5.x for myself. In fact, I wonder if it had to do with the updated POSIX standard since the standard lists each builtin it considers special. Who knows! – adam.hendry May 29 '21 at 18:20

1 Answers1

1

Because the user could have shadowed it with an alias or function, and using builtin bypasses those, but isn't completely portable to old shells.

tripleee
  • 175,061
  • 34
  • 275
  • 318