6

I'm into unit testing of some legacy shell scripts.

In the real world scripts are often used to call utility programs like find, tar, cpio, grep, sed, rsync, date and so on with some rather complex command lines containing a lot of options. Sometimes regular expressions or wildcard patterns are constructed and used.

An example: A shell script which is usually invoked by cron in regular intervals has the task to mirror some huge directory trees from one computer to another using the utility rsync. Several types of files and directories should be excluded from the mirroring process:

  #!/usr/bin/env bash
  ...
  function mirror() {
      ...
      COMMAND="rsync -aH$VERBOSE$DRY $PROGRESS $DELETE $OTHER_OPTIONS \
                   $EXCLUDE_OPTIONS $SOURCE_HOST:$DIRECTORY $TARGET"
      ...
      if eval $COMMAND
      then ...
      else ...
      fi
      ...
  }
  ...

As Michael Feathers wrote in his famous book Working Effectively with Legacy Code, a good unit test runs very fast and does not touch the network, the file-system or opens any database.

Following Michael Feathers advice the technique to use here is: dependency injection. The object to replace here is utility program rsync.

My first idea: In my shell script testing framework (I use bats) I manipulate $PATH in a way that a mockup rsync is found instead of the real rsync utility. This mockup object could check the supplied command line parameters and options. Similar with other utilities used in this part of the script under test.

My past experience with real problems in this area of scripting were often bugs caused by special characters in file or directory names, problems with quoting or encodings, missing ssh keys, wrong permissions and so on. These kind of bugs would have escaped this technique of unit testing. (I know: for some of these problems unit testing is simply not the cure).

Another disadvantage is that writing a mockup for a complex utility like rsync or find is error prone and a tedious engineering task of its own.

I believe the situation described above is general enough that other people might have encountered similar problems. Who has got some clever ideas and would care to share them here with me?

Potherca
  • 13,207
  • 5
  • 76
  • 94
pefu
  • 480
  • 4
  • 12
  • 3
    After unit testing, you need system testing. Maybe build a simple test network with one each of the problematic files, directories, missing keys, etc. – tripleee Mar 25 '16 at 11:50
  • 1
    Use `tee` inline with a real-world scenario to capture called command responses, then use those files to playback in the test harnessed scenario, perhaps with a script that logs the arguments received and blindly sends out the canned response. – Keith Tyler Mar 25 '16 at 19:09
  • @Keith Tyler: That is a really neat idea: Obviously this technique requires some sort of system test written beforehand (but when starting to fiddle on a legacy script, writing at least some such test before starting to refactor the script is anyway a good idea). Thumbs up for this suggestion. – pefu Mar 25 '16 at 21:32
  • When I have to execute complex, variable based commands, I usually execute it through a function similar to the following: `function echoExec { echo "$@" ; $@ }` When I test for the first few times, I can just comment out the execution line to make sure it would do what I expect. When the script is tested, I can comment out the echo line. Leaving both uncommented gives me a "verbose mode" if I need to debug the script in the future. – perogiex Apr 28 '16 at 18:05
  • "#!/usr/bin/env bash +x" bash start with debug mode. Also maybe you can use --dry-run rsync option. – Joao Vitorino May 25 '16 at 15:21
  • I haven't done it with shell, but elsewhere you instruct your mock to simulate external errors (i.e. network, filesystem, invalid input) and then test that your code is reacting to them properly. – Misha Tavkhelidze Sep 04 '16 at 08:31

2 Answers2

3

You can mockup any command using a function, like this:

function rsync() {
    # mock things here if necessary
}

Then export the function and run the unittest:

export -f rsync
unittest
hek2mgl
  • 152,036
  • 28
  • 249
  • 266
2

Cargill's quandary:

" Any design problem can be solved by adding an additional level of indirection, except for too many levels of indirection."

Why mock system commands ? After all if you are programming Bash, the system is your target goal and you should evaluate your script using the system.

Unit test, as the name suggests, will give you a confidence in a unitary part of the system you are designing. So you will have to define what is your unit in the case of a bash script. A function ? A script file ? A command ?

Given you want to define the unit as a function I would then suggest writing a list of well known errors as you listed above:

  • Special characters in file or directory names
  • Problems with quoting or encodings
  • Missing ssh keys
  • Wrong permissions and so on.

And write a test case for it. And try to not deviate from the system commands, since they are integral part of the system you are delivering.

Marco Silva
  • 323
  • 1
  • 6