Upon completion of this chapter, you will be able to Use for, while and until loops, and decide which loop fits which occasion. Use the break and continue Bash built-ins. Write scripts using the select statement. Write scripts that take a variable number of arguments. The for loop is the first of the three shell looping constructs. This loop allows for specification of a list of values. A list of commands is executed for each value in the list. The syntax for this loop is: for NAME [in LIST ]; do COMMANDS; done If [in LIST] is not present, it is replaced with in $@ and for executes the COMMANDS once for each positional parameter that is set (see and ). The return status is the exit status of the last command that executes. If no commands are executed because LIST does not expand to any items, the return status is zero. NAME can be any variable name, although i is used very often. LIST can be any list of words, strings or numbers, which can be literal or generated by any command. The COMMANDS to execute can also be any operating system commands, script, program or shell statement. The first time through the loop, NAME is set to the first item in LIST. The second time, its value is set to the second item in the list, and so on. The loop terminates when NAME has taken on each of the values from LIST and no items are left in LIST. The first is a command line example, demonstrating the use of a for loop that makes a backup copy of each .xml file. After issuing the command, it is safe to start working on your sources: [carol@octarine ~/articles] ls *.xml file1.xml file2.xml file3.xml [carol@octarine ~/articles] ls *.xml > list [carol@octarine ~/articles] for i in `cat list`; do cp "$i" "$i".bak ; done [carol@octarine ~/articles] ls *.xml* file1.xml file1.xml.bak file2.xml file2.xml.bak file3.xml file3.xml.bak This one lists the files in /sbin that are just plain text files, and possibly scripts: for i in `ls /sbin`; do file /sbin/$i | grep ASCII; done The following is a specific application script for converting HTML files, compliant with a certain scheme, to PHP files. The conversion is done by taking out the first 25 and the last 21 lines, replacing these with two PHP tags that provide header and footer lines: [carol@octarine ~/html] cat html2php.sh #!/bin/bash # specific conversion script for my html files to php LIST="$(ls *.html)" for i in "$LIST"; do NEWNAME=$(ls "$i" | sed -e 's/html/php/') cat beginfile > "$NEWNAME" cat "$i" | sed -e '1,25d' | tac | sed -e '1,21d'| tac >> "$NEWNAME" cat endfile >> "$NEWNAME" done Since we don't do a line count here, there is no way of knowing the line number from which to start deleting lines until reaching the end. The problem is solved using tac, which reverses the lines in a file. Instead of using sed to replace the html suffix with php, it would be cleaner to use the basename command. Read the man page for more info. You will run into problems if the list expands to file names containing spaces and other irregular characters. A more ideal construct to obtain the list would be to use the shell's globbing feature, like this: for i in $PATHNAME/*; do commands done The while construct allows for repetitive execution of a list of commands, as long as the command controlling the while loop executes successfully (exit status of zero). The syntax is: while CONTROL-COMMAND; do CONSEQUENT-COMMANDS; done CONTROL-COMMAND can be any command(s) that can exit with a success or failure status. The CONSEQUENT-COMMANDS can be any program, script or shell construct. As soon as the CONTROL-COMMAND fails, the loop exits. In a script, the command following the done statement is executed. The return status is the exit status of the last CONSEQUENT-COMMANDS command, or zero if none was executed. Here is an example for the impatient: #!/bin/bash # This script opens 4 terminal windows. i="0" while [ $i -lt 4 ] do xterm & i=$[$i+1] done The example below was written to copy pictures that are made with a webcam to a web directory. Every five minutes a picture is taken. Every hour, a new directory is created, holding the images for that hour. Every day, a new directory is created containing 24 subdirectories. The script runs in the background. #!/bin/bash # This script copies files from my homedirectory into the webserver directory. # (use scp and SSH keys for a remote directory) # A new directory is created every hour. PICSDIR=/home/carol/pics WEBDIR=/var/www/carol/webcam while true; do DATE=`date +%Y%m%d` HOUR=`date +%H` mkdir $WEBDIR/"$DATE" while [ $HOUR -ne "00" ]; do DESTDIR=$WEBDIR/"$DATE"/"$HOUR" mkdir "$DESTDIR" mv $PICDIR/*.jpg "$DESTDIR"/ sleep 3600 HOUR=`date +%H` done done Note the use of the true statement. This means: continue execution until we are forcibly interrupted (with kill or Ctrl+C). This small script can be used for simulation testing; it generates files: #!/bin/bash # This generates a file every 5 minutes while true; do touch pic-`date +%s`.jpg sleep 300 done Note the use of the date command to generate all kinds of file and directory names. See the man page for more. The previous example is for the sake of demonstration. Regular checks can easily be achieved using the system's cron facility. Do not forget to redirect output and errors when using scripts that are executed from your crontab! This script can be interrupted by the user when a Ctrl+C sequence is entered: #!/bin/bash # This script provides wisdom FORTUNE=/usr/games/fortune while true; do echo "On which topic do you want advice?" cat << topics politics startrek kernelnewbies sports bofh-excuses magic love literature drugs education topics echo echo -n "Make your choice: " read topic echo echo "Free advice on the topic of $topic: " echo $FORTUNE $topic echo done A here document is used to present the user with possible choices. And again, the true test repeats the commands from the CONSEQUENT-COMMANDS list over and over again. This script calculates the average of user input, which is tested before it is processed: if input is not within range, a message is printed. If q is pressed, the loop exits: #!/bin/bash # Calculate the average of a series of numbers. SCORE="0" AVERAGE="0" SUM="0" NUM="0" while true; do echo -n "Enter your score [0-100%] ('q' for quit): "; read SCORE; if (("$SCORE" < "0")) || (("$SCORE" > "100")); then echo "Be serious. Common, try again: " elif [ "$SCORE" == "q" ]; then echo "Average rating: $AVERAGE%." break else SUM=$[$SUM + $SCORE] NUM=$[$NUM + 1] AVERAGE=$[$SUM / $NUM] fi done echo "Exiting." Note how the variables in the last lines are left unquoted in order to do arithmetic. The until loop is very similar to the while loop, except that the loop executes until the TEST-COMMAND executes successfully. As long as this command fails, the loop continues. The syntax is the same as for the while loop: until TEST-COMMAND; do CONSEQUENT-COMMANDS; done The return status is the exit status of the last command executed in the CONSEQUENT-COMMANDS list, or zero if none was executed. TEST-COMMAND can, again, be any command that can exit with a success or failure status, and CONSEQUENT-COMMANDS can be any UNIX command, script or shell construct. As we already explained previously, the ; may be replaced with one or more newlines wherever it appears. An improved picturesort.sh script (see ), which tests for available disk space. If not enough disk space is available, remove pictures from the previous months: #!/bin/bash # This script copies files from my homedirectory into the webserver directory. # A new directory is created every hour. # If the pics are taking up too much space, the oldest are removed. while true; do DISKFUL=$(df -h $WEBDIR | grep -v File | awk '{print $5 }' | cut -d "%" -f1 -) until [ $DISKFUL -ge "90" ]; do DATE=`date +%Y%m%d` HOUR=`date +%H` mkdir $WEBDIR/"$DATE" while [ $HOUR -ne "00" ]; do DESTDIR=$WEBDIR/"$DATE"/"$HOUR" mkdir "$DESTDIR" mv $PICDIR/*.jpg "$DESTDIR"/ sleep 3600 HOUR=`date +%H` done DISKFULL=$(df -h $WEBDIR | grep -v File | awk '{ print $5 }' | cut -d "%" -f1 -) done TOREMOVE=$(find $WEBDIR -type d -a -mtime +30) for i in $TOREMOVE; do rm -rf "$i"; done done Note the initialization of the HOUR and DISKFULL variables and the use of options with ls and date in order to obtain a correct listing for TOREMOVE. Instead of controlling a loop by testing the result of a command or by user input, you can specify a file from which to read input that controls the loop. In such cases, read is often the controlling command. As long as input lines are fed into the loop, execution of the loop commands continues. As soon as all the input lines are read the loop exits. Since the loop construct is considered to be one command structure (such as while TEST-COMMAND; do CONSEQUENT-COMMANDS; done), the redirection should occur after the done statement, so that it complies with the form command < file This kind of redirection also works with other kinds of loops. In the example below, output of the find command is used as input for the read command controlling a while loop: [carol@octarine ~/testdir] cat archiveoldstuff.sh #!/bin/bash # This script creates a subdirectory in the current directory, to which old # files are moved. # Might be something for cron (if slightly adapted) to execute weekly or # monthly. ARCHIVENR=`date +%Y%m%d` DESTDIR="$PWD/archive-$ARCHIVENR" mkdir "$DESTDIR" # using quotes to catch file names containing spaces, using read -d for more # fool-proof usage: find "$PWD" -type f -a -mtime +5 | while read -d $'\000' file do gzip "$file"; mv "$file".gz "$DESTDIR" echo "$file archived" done Files are compressed before they are moved into the archive directory. The break statement is used to exit the current loop before its normal ending. This is done when you don't know in advance how many times the loop will have to execute, for instance because it is dependent on user input. The example below demonstrates a while loop that can be interrupted. This is a slightly improved version of the wisdom.sh script from . #!/bin/bash # This script provides wisdom # You can now exit in a decent way. FORTUNE=/usr/games/fortune while true; do echo "On which topic do you want advice?" echo "1. politics" echo "2. startrek" echo "3. kernelnewbies" echo "4. sports" echo "5. bofh-excuses" echo "6. magic" echo "7. love" echo "8. literature" echo "9. drugs" echo "10. education" echo echo -n "Enter your choice, or 0 for exit: " read choice echo case $choice in 1) $FORTUNE politics ;; 2) $FORTUNE startrek ;; 3) $FORTUNE kernelnewbies ;; 4) echo "Sports are a waste of time, energy and money." echo "Go back to your keyboard." echo -e "\t\t\t\t -- \"Unhealthy is my middle name\" Soggie." ;; 5) $FORTUNE bofh-excuses ;; 6) $FORTUNE magic ;; 7) $FORTUNE love ;; 8) $FORTUNE literature ;; 9) $FORTUNE drugs ;; 10) $FORTUNE education ;; 0) echo "OK, see you!" break ;; *) echo "That is not a valid choice, try a number from 0 to 10." ;; esac done Mind that break exits the loop, not the script. This can be demonstrated by adding an echo command at the end of the script. This echo will also be executed upon input that causes break to be executed (when the user types 0). In nested loops, break allows for specification of which loop to exit. See the Bash info pages for more. The continue statement resumes iteration of an enclosing for, while, until or select loop. When used in a for loop, the controlling variable takes on the value of the next element in the list. When used in a while or until construct, on the other hand, execution resumes with TEST-COMMAND at the top of the loop. In the following example, file names are converted to lower case. If no conversion needs to be done, a continue statement restarts execution of the loop. These commands don't eat much system resources, and most likely, similar problems can be solved using sed and awk. However, it is useful to know about this kind of construction when executing heavy jobs, that might not even be necessary when tests are inserted at the correct locations in a script, sparing system resources. [carol@octarine ~/test] cat tolower.sh #!/bin/bash # This script converts all file names containing upper case characters into file# names containing only lower cases. LIST="$(ls)" for name in "$LIST"; do if [[ "$name" != *[[:upper:]]* ]]; then continue fi ORIG="$name" NEW=`echo $name | tr 'A-Z' 'a-z'` mv "$ORIG" "$NEW" echo "new name for $ORIG is $NEW" done This script has at least one disadvantage: it overwrites existing files. The noclobber option to Bash is only useful when redirection occurs. The -b option to the mv command provides more security, but is only safe in case of one accidental overwrite, as is demonstrated in this test: [carol@octarine ~/test] rm * [carol@octarine ~/test] touch test Test TEST [carol@octarine ~/test] bash -x tolower.sh ++ ls + LIST=test Test TEST + [[ test != *[[:upper:]]* ]] + continue + [[ Test != *[[:upper:]]* ]] + ORIG=Test ++ echo Test ++ tr A-Z a-z + NEW=test + mv -b Test test + echo 'new name for Test is test' new name for Test is test + [[ TEST != *[[:upper:]]* ]] + ORIG=TEST ++ echo TEST ++ tr A-Z a-z + NEW=test + mv -b TEST test + echo 'new name for TEST is test' new name for TEST is test [carol@octarine ~/test] ls -a ./ ../ test test~ The tr is part of the textutils package; it can perform all kinds of character transformations. The select construct allows easy menu generation. The syntax is quite similar to that of the for loop: select WORD [in LIST]; do RESPECTIVE-COMMANDS; done LIST is expanded, generating a list of items. The expansion is printed to standard error; each item is preceded by a number. If in LIST is not present, the positional parameters are printed, as if in $@ would have been specified. LIST is only printed once. Upon printing all the items, the PS3 prompt is printed and one line from standard input is read. If this line consists of a number corresponding to one of the items, the value of WORD is set to the name of that item. If the line is empty, the items and the PS3 prompt are displayed again. If an EOF (End Of File) character is read, the loop exits. Since most users don't have a clue which key combination is used for the EOF sequence, it is more user-friendly to have a break command as one of the items. Any other value of the read line will set WORD to be a null string. The read line is saved in the REPLY variable. The RESPECTIVE-COMMANDS are executed after each selection until the number representing the break is read. This exits the loop. This is a very simple example, but as you can see, it is not very user-friendly: [carol@octarine testdir] cat private.sh #!/bin/bash echo "This script can make any of the files in this directory private." echo "Enter the number of the file you want to protect:" select FILENAME in *; do echo "You picked $FILENAME ($REPLY), it is now only accessible to you." chmod go-rwx "$FILENAME" done [carol@octarine testdir] ./private.sh This script can make any of the files in this directory private. Enter the number of the file you want to protect: 1) archive-20030129 2) bash 3) private.sh #? 1 You picked archive-20030129 (1) #? Setting the PS3 prompt and adding a possibility to quit makes it better: #!/bin/bash echo "This script can make any of the files in this directory private." echo "Enter the number of the file you want to protect:" PS3="Your choice: " QUIT="QUIT THIS PROGRAM - I feel safe now." touch "$QUIT" select FILENAME in *; do case $FILENAME in "$QUIT") echo "Exiting." break ;; *) echo "You picked $FILENAME ($REPLY)" chmod go-rwx "$FILENAME" ;; esac done rm "$QUIT" Any statement within a select construct can be another select loop, enabling (a) submenu(s) within a menu. By default, the PS3 variable is not changed when entering a nested select loop. If you want a different prompt in the submenu, be sure to set it at the appropriate time(s). The shift command is one of the Bourne shell built-ins that comes with Bash. This command takes one argument, a number. The positional parameters are shifted to the left by this number, N. The positional parameters from N+1 to $# are renamed to variable names from $1 to $# - N+1. Say you have a command that takes 10 arguments, and N is 4, then $4 becomes $1, $5 becomes $2 and so on. $10 becomes $7 and the original $1, $2 and $3 are thrown away. If N is zero or greater than $#, the positional parameters are not changed (the total number of arguments, see ) and the command has no effect. If N is not present, it is assumed to be 1. The return status is zero unless N is greater than $# or less than zero; otherwise it is non-zero. A shift statement is typically used when the number of arguments to a command is not known in advance, for instance when users can give as many arguments as they like. In such cases, the arguments are usually processed in a while loop with a test condition of (( $# )). This condition is true as long as the number of arguments is greater than zero. The $1 variable and the shift statement process each argument. The number of arguments is reduced each time shift is executed and eventually becomes zero, upon which the while loop exits. The example below, cleanup.sh, uses shift statements to process each file in the list generated by find: #!/bin/bash # This script can clean up files that were last accessed over 365 days ago. USAGE="Usage: $0 dir1 dir2 dir3 ... dirN" if [ "$#" == "0" ]; then echo "$USAGE" exit 1 fi while (( "$#" )); do if [[ $(ls "$1") == "" ]]; then echo "Empty directory, nothing to be done." else find "$1" -type f -a -atime +365 -exec rm -i {} \; fi shift done The above find command can be replaced with the following: find options | xargs [commands_to_execute_on_found_files] The xargs command builds and executes command lines from standard input. This has the advantage that the command line is filled until the system limit is reached. Only then will the command to execute be called, in the above example this would be rm. If there are more arguments, a new command line will be used, until that one is full or until there are no more arguments. The same thing using find -exec calls on the command to execute on the found files every time a file is found. Thus, using xargs greatly speeds up your scripts and the performance of your machine. In the next example, we modified the script from so that it accepts multiple packages to install at once: #!/bin/bash if [ $# -lt 1 ]; then echo "Usage: $0 package(s)" exit 1 fi while (($#)); do yum install "$1" << CONFIRM y CONFIRM shift done In this chapter, we discussed how repetitive commands can be incorporated in loop constructs. Most common loops are built using the for, while or until statements, or a combination of these commands. The for loop executes a task a defined number of times. If you don't know how many times a command should execute, use either until or while to specify when the loop should end. Loops can be interrupted or reiterated using the break and continue statements. A file can be used as input for a loop using the input redirection operator, loops can also read output from commands that is fed into the loop using a pipe. The select construct is used for printing menus in interactive scripts. Looping through the command line arguments to a script can be done using the shift statement. Remember: when building scripts, work in steps and test each step before incorporating it in your script. Create a script that will take a (recursive) copy of files in /etc so that a beginning system administrator can edit files without fear. Write a script that takes exactly one argument, a directory name. If the number of arguments is more or less than one, print a usage message. If the argument is not a directory, print another message. For the given directory, print the five biggest files and the five files that were most recently modified. Can you explain why it is so important to put the variables in between double quotes in the example from ? Write a script similar to the one in , but think of a way of quitting after the user has executed 3 loops. Think of a better solution than move -b for the script from to prevent overwriting of existing files. For instance, test whether or not a file exists. Don't do unnecessary work! Rewrite the whichdaemon.sh script from , so that it: Prints a list of servers to check, such as Apache, the SSH server, the NTP daemon, a name daemon, a power management daemon, and so on. For each choice the user can make, print some sensible information, like the name of the web server, NTP trace information, and so on. Optionally, build in a possibility for users to check other servers than the ones listed. For such cases, check that at least the given process is running. Review the script from . Note how character input other than q is processed. Rebuild this script so that it prints a message if characters are given as input.