Why should I learn Shell Programming at all? What can be done with it in the real world? Could you show me some powerful things that can be done with it or some special features so as to convince me that I should start learning shell programming right now?
Asked
Active
Viewed 2.0k times
6 Answers
19
There are a billion and one reasons to learn shell programming. One of the major reasons is systems administration.
Here is an example of somebody who needed to rename ~750 files based upon another file in that same directory. This was accomplished with 3 lines of shell scripting. To get many more examples just search for questions with the tags [bash], [sed] or [awk].
When somebody asks me to show them a cool example of "shell programming" I always show them this awk 1-liner (maybe even a 1-worder?). It will filter a list to only show unique values without changing the original order. This is significant because most other solutions require you to sort the list first which destroys the original order.
$ echo -e "apple\npear\napple\nbanana\nmango\npear\nbanana" | awk '!a[$0]++'
apple
pear
banana
mango
Explanation of awk command
The non-sorting unique magic happens with !a[$0]++. Since awk supports associative arrays, it uses the current record (aka line) $0 as the key to the array a[]. If that key has not been seen before, a[$0] evaluates to 0 (zero) which is awk's default value for unset indices. We then negate this value to return TRUE on the first occurrence of that key. a[$0] is then incremented such that subsequent hits on this key will return FALSE and thus repeat values are never printed. We also exploit the fact that awk will default to print $0 (print the current record/line) if an expression returns TRUE and no further { commands } are given.
If you still don't understand, don't worry, this is a very terse and optimized version of what could be a much longer awk script.
-
-e is for evaluating special chars like "\n"'s, that much i get. You passed a string to echo with that option. but what is the '|' and !a[$0]++ etc. could you introduce those to me? – fabio Dec 16 '10 at 19:05
-
`|` is a pipe. It means that the *output* (stdout) of the left-hand-side command should be used as *input* (stdin) of the right-hand-side command. In this particular case I'm using the pipe to tell awk to use the output of `echo` as the input text to work on. I'll explain the `awk '!a[$0]++'` in my answer since I'm going to need more space =) – SiegeX Dec 16 '10 at 19:06
-
thanks a lot, I read you explanation, it's a very short and beautiful code doing lots of things at once... i'm guessing that when you use pipe the string is automatically broken by the "\n" chars, so it becomes an array with several elements, and then you loop through them, i just don't see where the looping is happening!! – fabio Dec 16 '10 at 19:26
-
@user not so much an array but input with multiple lines, this is done by `echo` not the pipe. The pipe's job is just to connect stdout of echo to stdin of awk. You don't see any loop because of the way awk works. awk, by default, will run the commands you give it on each line passed in. – SiegeX Dec 16 '10 at 19:29
-
wow, amazing i'm getting it!!! I think it's cool how you solve the problem of getting only the first time the element appears, that is, only when ZERO it becomes true (because !0==true) and print it, the other times it doesn't (!1==false,!2==false etc), really beautiful!!! – fabio Dec 16 '10 at 19:31
-
i think the most experienced shell programmer wouldn't be able to come with such a solution so quickly for solving a problem, unless he/she had already used this previously, like you, or am I wrong? – fabio Dec 16 '10 at 19:42
-
@user yes to your question, and I'm a shining example of this. I cannot take credit for this awk script but I, like you, appreciated the beauty in its simplicity yet powerfulness when I fully understood it. P.S. welcome to SO – SiegeX Dec 16 '10 at 20:18
8
Dude, you don't need to learn shell programming right now if you don't want to. But if you want to actually do things on your computer, it can come in handy.
Pretty much any kind of computer will have its system scripts written in its native shell code. So, if you want to understand and configure what it does, you will need to learn the shell script.
The other useful thing is that it will teach you how to use your command line properly. You can't do everything with tools (e.g., cp -r foo bar) or simple globbing (e.g., rm *.o); even something as simple as mass renaming will require a loop. And shell is the simplest and most appropriate tool for that kind of job.
comingstorm
- 25,557
- 3
- 43
- 67
5
I needed an alarm the other day.
sleep 20m && madplay song.mp3
When you know shell programming, you find uses for it that simplify your life. The example before was simple, more fun than useful, but it can save you tons of time in filesystem operations and a miriad other things. A google query can give you an idea.
salezica
- 74,081
- 25
- 105
- 166
-
-
Wouldn't really know, but the Linux mostly-everything-is-a-file philosophy gives shell programming great power. Besides, there are amazing command-line utilities for Linux, that can be piped and redirected to work together, and I doubt that's the case with Windows. – salezica Dec 16 '10 at 19:22
1
Shell programming can be very useful for some quick one-line commands that can still do a lot. The big advantage is that you don't have to save the code.
However, you could easily replace shell programming by, for example, using the python command line.
Tom van der Woerdt
- 29,532
- 7
- 72
- 105
1
The first thing most people will use shell scripting for is automating a simple task, like backups. My first script was just playing around with Rsync. Then called the script with cron.
Other fun things: Maybe you need to search a bunch of text and perform a find-and-replace in multiple files. Or perhaps you want to keep a tarball of your home directory as it existed yesterday. Or any other redundant thing you do in shell.
themicahmachine
- 166
- 2
-
That find-and-replace thing was something i always wanted to do and had to use the find-and-replace engine of other programs to do the task for me... but there were cases where i couldn't do more serious things due to there limitations... i'm almost convinced!! – fabio Dec 16 '10 at 19:10
1
Shell programming lets you slice and dice tasks that would be painfully slow if you were to use the GUI.
The Pragmatic Programmer by Hunt and Thomas discusses this in fair detail.
John
- 15,990
- 10
- 70
- 110