10

When Bash runs your script, does it parse it as code, or does it parse it as a command? Does Bash actually compile, and run your script like Python would do, or does Bash just run it through its command parser?

codeforester
  • 39,467
  • 16
  • 112
  • 140
Jon .C
  • 139
  • 1
  • 6
  • 3
    What scripting languages compile their code? – PM 77-1 Jan 20 '17 at 02:54
  • 5
    @PM77-1: Python, for one. – paxdiablo Jan 20 '17 at 02:57
  • 1
    ruby, python, etc. it doesn't really "compile" it into a file, but it takes the file, and writes it in compiled language that is transferred into machine code en.wikipedia.org/wiki/Scripting_language sort of like a temp compile . Image in if you could take c as a .c file with plain text then gcc took it and compiled it into a temp file ,and ran that file. That is basically what a scripting language does. – Jon .C Jan 20 '17 at 03:10
  • Perl "precompiles" a parse tree - gets as much done as possible at initialization, and *can* save that version to "precompiled" copy to speed things along in some cases. I'd assume Groovy uses some JIT bytecode generation. I think Bash is truly line-at-a-time interpreter, though, yes? – Paul Hodges Dec 06 '21 at 14:25
  • I do know that one version of the HPUX `ksh` interpreter I used years ago would preread in the whole file into a buffer, while another just moved a pointer long in the file; that meant that for the *second* one, you could run the file and make a change early in the file, and the running copy would get confused because the pointer now wasn't pointing at the beginning of a line, since you had changed the offsets - the other version kept running the cached copy. Both have advantages and disadvantages, but I prefer the caching. – Paul Hodges Dec 06 '21 at 14:28

3 Answers3

19

Bash is a single-pass interpreter which means it reads one command at a time, interprets, and runs it then and there. The same thing is true with other types of shells - sh, ksh, zsh, csh, etc.

Here is an example. I have a 3 line script called test.sh which looks like this:

echo one
echo two
'

When run as bash test.sh, it gives this output:

one
two
test.sh: line 3: unexpected EOF while looking for matching `''
test.sh: line 4: syntax error: unexpected end of file

It runs the first and second commands successfully and then encounters the dangling single quote and throws the error.

Let's say we write the same code in Perl, test.pl:

print "one\n"
print "two\n"
'

and run it with perl test.pl. We get:

syntax error at test.pl line 2, near "print"
Can't find string terminator "'" anywhere before EOF at test.pl line 3.

So, it didn't run the first two lines at all, though they were syntactically correct. That's because Perl makes two passes. In the first pass, it does syntax checks and converts the script into an internal form. In the second pass, it runs it.

The simplicity of shell's single-pass execution is its biggest limitation as well. Tolerating syntax errors, even running at all, makes it hard to build large and robust code with the shell language. However, shell scripting is an ideal choice for quick and throw-away code, especially something that makes use of a lot of command line utilities.

Related:

codeforester
  • 39,467
  • 16
  • 112
  • 140
  • 1
    Fun fact: you can even modify script while interpreter (bash) is running, and changes will be in effect: `echo "Hello"; echo "echo World" >> $0` – el.pescado - нет войне Oct 11 '18 at 06:34
  • This is simply not true _"so, it didn't run the first two lines at all, though they were syntactically correct. That's because Perl makes two passes. In the first pass, it does syntax checks and converts the script into an internal form. In the second pass, it runs it."_ Perl is a single pass compiler. It makes one pass, compiles everything into an optree (a kind of an AST and opcode), and hands it off to the Perl VM which executes the optree. It does not make another pass. Now in the case of `eval` the parser can be reinvoked and modify the optree, but that's another seperate thing. – Evan Carroll Jun 12 '22 at 20:39
  • To put it another way, perl is a late binding compiler. Not a double-pass interpreter. And because of this [some things are kinda weird](https://stackoverflow.com/a/70918283/124486). – Evan Carroll Jun 12 '22 at 20:40
9

The bash shell, as it stands now (version 4.4), runs your scripts in a purely textual fashion. It does not do any pre-compilation of files into some form of byte code.

As per the source code, the shell itself just uses reader_loop() to process the input. This calls, within a loop, read_command() followed by execute_command().

As the read_command() also contains the call to parse_command() which calls the YACC parser function yyparse(), this means that parsing happens on a line-by-line basis, not up front in some large compilation phase.

paxdiablo
  • 854,327
  • 234
  • 1,573
  • 1,953
  • 8
    This is why variable name length can actually noticeably impacts performance in bash, as opposed to compiled or JIT-based languages. e.g. `time for ((i=0;i<100000;++i)); do :; done` will be much slower if you rename `i` something longer. – Jeffrey Cash Jan 20 '17 at 03:13
3

Another way to see that bash processes its input one line at a time is to create a script with an obvious syntax error after a valid echo statement, then run the script. bash will produce some output before producing a syntax error:

echo output
()

produces

$ bash tmp.sh
output
tmp.sh: line 2: syntax error near unexpected token ')'
tmp.sh: line 2: `()'
chepner
  • 497,756
  • 71
  • 530
  • 681