drm's tweakblog

I've found bash scripting to be one of the gems I (re)discovered this year. When you're like me, you like to automate stuff. There's of course loads of scripting languages available to get it done, but frankly, using bash for lots of filesystem and/or configuration related tasks makes more sense than for example PHP or Python.

Here are some pointers to get you on track. As always feel free to ask questions.

Streams and redirection
First of all, you need to know (and understand) what streams are, and how you connect the output of one program to the input of another. For this, you'll use pipes. A pipe connects the output of a program to the input of the other.

Simple example:

The "SELECT 1" string is input for the mysql program, so mysql will execute the query.

Then there's redirection. You have output redirection, and input redirection. You can use this as follows:


Output redirection is mostly used to ignore output of scripts by writing errors and output to /dev/null (the black hole of nothingness), input redirection is not that common.

Tests
You can use if for tests. The basic structure is

code:

1 2 3 4 5 6 7 8 9

if condition then # do something elif someothercondition then # do something else else # do even more stuff fi

You can shorthand the code by inserting a semi-colon ; wherever there's a newline, except after "then", so would work fine. I prefer writing the "if" and the "then" on one line. There are lots of tests and evaluations available, but the most commonly used are the following:

• [ -d "dir" ] check if "dir" is a directory
• [ -f "file" ] check if "file" is a file
• ( program ) See if program runs succesfully (i.e. doesn't return an exit code)
• [ "$var" == "" ] check if $var is empty.

Further reading about test constructs.

case statements
These provide similar constructs to switch statements in e.g. PHP or C.

code:

1 2 3 4

case $1 in "a" ) echo "We'll do a";; * ) echo "Ok we'll do nothing";; esac

Note the double semicolons terminating each case. These are particularly useful for check for simple arguments.

For-loops
The most common loop you would use are for-loops. With for loops you can walk through a collection of items, separated by space or newline.
prints:

a
b
c

Back-ticks and evaluations
You can use back-tick quotes to evaluate a piece of script to a string. This can be as simple as saving a variable.
Example

Or using it in a for loop:
prints every file in the current directory. Which is useless, of course

You can also use $( ... ) to evaluate a command and use that in a for loop, which makes it easier to nest evaluations


xargs
xargs is used to use input as a parameter for the given command. Examples explain best, so:
Does about the same as the previous script.

find
Find is probably the most powerful tool in shell scripting. It finds files for you, based on almost any condition you can think of. The most commonly used though, are by type, and by name, or a combination of both. The first argument is always the directory where you will be starting your search.
Examplbe:

A very commonly used option is -print0, to print the results terminated with a \0 character in stead of a newline. This is useful in conjunction with xarg's -0 option, when the results contain spaces, so the filenames aren't cut in half while processing.

Read the find man pages for more information, or just type

The processing commands
cat Simply output a file's contents.


grep: find a certain pattern in a string or a file, and print matching lines.
Mostly used options: -n (print line number), -e (use regular expressions), -o (only print matching part)
Examples


awk: An inline scripting language to process text.
Mostly used: awk '{print $2}', where '2' is a column number
Example


sed: Stream editor, process input and render output.
Mostly used: sed 's/replace this/with that/' in a pipe


cut: Cut parts of an input string for each line
Mostly used: cut -c -5, cut -c 5, cut -c 5-, cuts the last 5, from the 5th, and the first 5 of a string respectively and prints the rest.
Prints only the files, not the status.
This is especially useful if you can't trust the output to be space-separated columns, to use with awk.

A sample script file
A script file is a text file, containing a first line pointing to the interpreter (the shebang)
Or for freebsd users: and the following lines are just simply executes as if it were command lines.

You can use numbered arguments, with $0 pointing to the script file itself. So here's a little sample file giving you an impression of how to use that:



Now, go combine, mix, and match, and most of all, learn more on the way. Read more about bash scripting on tldp.org and get comfortable with the bash readline shortcuts, consider downloading and printing a cheatsheet. And don't forget the man pages for built-in shell commands.

Final tip: use forward slash to search inside a manpage and 'n' to scroll to the next search match.

In my experience, the easiest problems aren't easy to solve by simply googling, so if you run into anything don't hesitate to ask here.