4

I am currently working on some scripts that automate a lot of stuff and are fairly medium sized. I have started to follow certain things to make my scripts generic and portable like:

  • administrator login for scheduling scripts
  • use of absolute paths
  • use of config files
  • use write-host when developing and switch to logging when deploying (debugging)

I would be most interested in knowing if there are any standard scripting techniques and guidelines out there that aim to make a script developers life simple. I work on both perl and powershell, so a more generalized advice would be most welcome and appreciated.

Thanks for any help in advance.
steeluser

Emiliano Poggi
  • 24,390
  • 8
  • 55
  • 67
Animesh
  • 4,926
  • 14
  • 68
  • 110
  • When you say 'use of absolute paths', do you mean 'you use absolute paths' or 'you avoid using absolute paths'? The former tends to lead to non-portability; the latter to portability. – Jonathan Leffler May 04 '11 at 06:31
  • I meant use of absolute paths is better. I am thinking this because my script has to work flawlessly without throwing error wherever it gets called from. I am not sure I understand how it leads to non-portability. Could you please elaborate a bit more on this? Thanks. – Animesh May 04 '11 at 06:37
  • It means that every machine has to have everything in the same place. As soon as something else is where you wanted to put your script, or there is no space left for your script, then you have a major problem on your hands. – Jonathan Leffler May 04 '11 at 06:52
  • This makes a lot of sense to me to not use the absolute paths. Thanks for the insight. – Animesh May 04 '11 at 12:11

2 Answers2

6

This is kind of broad and I'm sure subjective. However, I've done some scripting work and can offer some things I've learnt from experience.

  1. Make your scripts data driven. Hardcode as few things as possible. All things like paths, resource locations etc. should be changeable from a config file rather than by editing the main script. This ties in with your point about absolute paths. Hardcoding all the paths is a no no. Making everything relative imposes too many constraints. It would be nice to have a single variable called, for example, DATA_ROOT which you can modify and then have all the routines refer to things like ${DATA_ROOT}/resources/images etc. A ${PROGRAM_ROOT} would be useful too so that you can have multiple installations of your suite to test etc.
  2. Error checking. Scripting languages are often dynamically typed so make sure you check the return value of every thing you do. It might even make sense to make a library with which you can do things like call_safely(fn, arg0, arg1,.. ) and call_and_abort_on_failure(fn, arg0, arg1,...).
  3. Comment and document liberally. Scripts are easier to modify that compiled programs and so they're often patched in the wrong ways when people want to extend them. Make sure that your APIs are well documented so that when people want to, for example, add a new command line option, they know how to do it rather than write a wrapper script on top of yours.
  4. Make one script do one thing and write aggregators if you need. scripts that have if(FLAG) {do_something} else {do_something_completely_different} will kill you. Make your scripts do one thing and do it well and if you want, you can write wrappers that delegate to these scripts as necessary (for a better user experience).
  5. Use 3rd party tools liberally rather than invent your own. Command line parsing, text processing etc. are all old areas and there are lots of libraries that do them well. Use those rather than reimplement. The most bug free program is the one that's never written.
  6. Make your logging and testing frameworks rock solid. When bad things happen, you should know everything without having to rerun the scripts. Use some standard logging framework from day one and make it configurable from a file which can be reloaded on demand. Make sure that your code is unit tested so that when new changes are made, regressions bugs are not introduced. Also, log timing metrics so that you know over time where your scripts are slowing down. It will be useful in the future.
  7. Make your scripts as tinker free as possible. This is hard to describe but I can give you an example. I had some scripts to track various metrics on builds for various branches and drop them into a directory. I wrote it so that if you wanted a branch foo tracked, all you had to do was to create a directory called foo in a certain place and it would track that. You didn't have to tinker with a lot of stuff and the managers who actually wanted to track metrics could create the directories as they needed it.

These are the points I can think of from the top of my head.

Noufal Ibrahim
  • 71,383
  • 13
  • 135
  • 169
  • All your points are really good. I am not sure why stuff like if($productionmode -ne 1) {dont send mail} would kill my scripts. Could you please elaborate on this? – Animesh May 04 '11 at 12:08
  • 2
    Because your code would become completely littered with `if`'s and `else`'s and will become hard to read and unmaintainable. If you have a notification framework which sends out emails that whose configuration you can tweak from the command line or a config file, it will be much more manageable than the ifs and elses. – Noufal Ibrahim May 04 '11 at 12:12
1

just a few practices which I consider good:

  • Using absolute paths is ok as far as you catch them at runtime. See this question.
  • Use for configuration files some easily processing format such as xml or yaml.
  • DO NOT use write-host, use better write-output or write-log or, better, implement a custom function which allows you to switch between console/log according to a setting on your configuration files. See my answer to this question for any more details.

These are just a few suggestions based on your points.

EDIT Note about adopting XML configuration file

Use XML as far as your configuration file will contain a few settings and you are going to manage it only via script.

Hope this help

Cheers

Community
  • 1
  • 1
Emiliano Poggi
  • 24,390
  • 8
  • 55
  • 67
  • YAML format looks good. Do you know if there is something like import-yaml for powershell? – Animesh May 04 '11 at 12:08
  • Not really. Try codeplex or poshcode. I think this is a good new question around powershell tag. However I would first have a look to .NET. Try [this](http://stackoverflow.com/questions/42071/net-yaml-library). – Emiliano Poggi May 04 '11 at 12:21
  • 2
    Please. Not XML. It will grow over time and become very hard to edit and manage by hand. This *is* a human written config file. Not a computer generated and computer consumed data set. – Noufal Ibrahim May 04 '11 at 14:54
  • @Noufal: Yes, you are true. Edited the question for improvement. – Emiliano Poggi May 04 '11 at 15:08