LDP/LDP/howto/linuxdoc/Bash-Prog-Intro-HOWTO.sgml

<!doctype linuxdoc system>

<!-- 
     An introduction to bash shell programming
 -->

<!-- Title information -->

<article>
    <title>BASH Programming - Introduction HOW-TO</title>
    <author>by Mike G <tt/mikkey@dynamo.com.ar/</author>
    <date>v0.05, 20 June 2000</date>
    <abstract> 
    This article intends to help you to start programming 
    basic-intermediate shell scripts. It does not intend to be an 
    advanced document (see the title). I am NOT an expert nor guru 
    shell programmer. I decided to write this because I'll learn a 
    lot and it might be useful to other people. Any feedback will be apreciated, 
    specially in the patch form :) 
    </abstract>

<!-- Table of contents -->
<toc>

<!-- Requisites -->
<sect>
Introduction

<!--
     <sect1>
     Getting the latest version

     <P> View it online
     <htmlurl url=""
     name="">


     </sect1>
-->
     <sect1>
     Requisites
          <P> Familiarity with GNU/Linux command lines, and familiarity 
          with basic programming concepts is helpful. While this is not
          a programming introduction, it explains (or at least tries) many
          basic concepts.
          <P> 
     </sect1>
     <sect1>
     Uses of this document
          <P> This document tries to be useful in the following situations
	  <itemize>
	      <item> You have an idea about programming and you want to start 
	      coding some shell scripts.
	      <item> You have a vague idea about shell programming and want 
	      some sort of reference.
	      <item> You want to see some shell scripts and some comments to
	      start writing your own
	      <item> You are migrating from DOS/Windows (or already did) 
	      and want to make "batch" processes.
	      <item> You are a complete nerd and read every how-to available
	  </itemize>
     </sect1>
</sect>

<!-- Very simple Scripts --> 
<sect>
Very simple Scripts

    <P> This HOW-TO will try to give you some hints about shell script
    programming strongly based on examples.
    <P> In this section you'll find some little scripts which 
    will hopefully help you to understand some techniques.

    <sect1>
    Traditional hello world script

        <!-- __TO-DO__
	someone suggested using this instead of the line numbering 
	but it doesn't work on verion 1.0.9
	<pre></pre> 
	-->
	<P>
	<tscreen><verb>
	  #!/bin/bash          
	  echo Hello World!    
	</verb></tscreen>
	<P> 
	<P> This script has only two lines. 
        The first indicates the system which program 
        to use to run the file. 
        <P> The second line is the only action performed by this script, 
        which prints 'Hello World! on the terminal.
    </sect1>
    <sect1>
    A very simple backup script
        <P>
	<tscreen><verb>
	#!/bin/bash          
	tar -cZf /var/my-backup.tgz /home/me/
	</verb></tscreen>
        <P> In this script, instead of printing a message on the terminal, 
	we create a tar-ball of a user's home directory. This is NOT intended
	to be used, a more useful backup script is presented later in this 
        document.
    </sect1>   
</sect>

<!-- Redirection -->
<sect>
All about redirection

	<sect1>
	Theory and quick reference
	<p> There are 3 file descriptors, stdin, stdout and stderr (std=standar)

	<p>Basically you can:
	<enum>
	<item> redirect stdout to a file
	<item> redirect stderr to a file
	<item> redirect stdout to a stderr 
	<item> redirect stderr to a stdout 
	<item> redirect stderr and stdout to a file 
	<item> redirect stderr and stdout to stdout 
	<item> redirect stderr and stdout to stderr
	</enum>
	1 'represents' stdout and 2 stderr
	<p> A little note for seeing this things: with the less command you can view both stdout 
	(which will remain on the buffer) and the stderr that will be printed on the screen, but erased as 
	you try to 'browse' the buffer. 
	</sect1>
	
	<sect1>
	Sample: stdout 2 file 
	<p> This will cause the ouput of a program to be writen to a file
	<tscreen><verb>
	ls -l > ls-l.txt
	</verb></tscreen>
	Here, a file called 'ls-l.txt' will be created and it will contain what you would see on the 
	screen if you type the command 'ls -l' and execute it.
	</sect1>

	<sect1>
	Sample: stderr 2 file 
	<p> This will cause the stderr ouput of a program to be writen to a file
	<tscreen><verb>
	grep da * 2> grep-errors.txt
	</verb></tscreen>
	Here, a file called 'grep-errors.txt' will be created and it will contain what you would see
	the stderr portion of the output of the 'grep da *' command.
	</sect1>

	<sect1>
	Sample: stdout 2 stderr
	<p> This will cause the stderr ouput of a program to be writen to the same filedescriptor 
	than stdout.
	<tscreen><verb>
	grep da * 1>&2 
	</verb></tscreen>
	Here, the stdout portion of the command is sent to stderr, you may notice that in differen ways.
	</sect1>

	<sect1>
	Sample: stderr 2 stdout 
	<p> This will cause the stderr ouput of a program to be writen to the same filedescriptor 
	than stdout.
	<tscreen><verb>
	grep * 2>&1
	</verb></tscreen>
	Here, the stderr portion of the command is sent to stdout, if you pipe to less, you'll see that
	lines that normally 'dissapear' (as they are written to stderr) are being kept now (because
	they're on stdout). 
	</sect1>

	<sect1>
	Sample: stderr and stdout 2 file 
	<p> This will place every output of a program to a file. This is suitable sometimes 
	for cron entries, if you want a command to pass in absolute silence.  
	<tscreen><verb>
	rm -f $(find / -name core) &> /dev/null 
	</verb></tscreen>
	This (thinking on the cron entry) will delete every file called 'core' in any directory. Notice
	 that you should be pretty sure of what a command is doing if you are going to wipe it's output. 
	</sect1>
<!--
redirect stderr and stdout to stdout 
redirect stderr and stdout to stderr
-->



<!--
	<sect1>
	Sample: 
	<p> This will 
	<tscreen><verb>
	command
	</verb></tscreen>
	Explanation
	</sect1>
-->

</sect>

<!-- pipes -->
<sect>
Pipes
	<p> This section explains in a very simple and practical way how to use pipes, 
	nd why you may want it.

	<sect1>
	What they are and why you'll want to use them
		<p> Pipes let you use (very simple, I insist) the output of a program as the 
		input of another one	
	</sect1>


	<sect1>
	Sample: simple pipe with sed 
	<p> This is very simple way to use pipes.
	<tscreen><verb>
	ls -l | sed -e "s/[aeio]/u/g"	
	</verb></tscreen>
	Here, the following happens: first the command ls -l is executed, and it's output, 
	instead of being printed, is sent (piped) to the sed program, which in turn, prints
	what it has to. 
	</sect1>

	<sect1>
	Sample: an alternative to ls -l *.txt 
	<p> Probably, this is a more difficult way to do ls -l *.txt, but it is here for illustrating pipes,
	not for solving such listing dilema. 
	<tscreen><verb>
	ls -l | grep "\.txt$"
	</verb></tscreen>
	Here, the output of the program ls -l is sent to the grep program, which, in turn, will print 
	lines which match the regex "\.txt$".
	</sect1>



<!--
	<sect1>
	Sample: 
	<p> This will 
	<tscreen><verb>
	command
	</verb></tscreen>
	Explanation
	</sect1>
-->


</sect>




<!-- Variables -->
<sect>
Variables

    <!-- Dry Theory -->
        <P> You can use variables as in any programming languages. 
	There are no data types. A variable in bash can contain a number, a
	character, a string of characters. 
	<P>  You have no need to declare a variable, just 
        assigning a value to its reference will create it.


    <!-- Samples -->
        <sect1>
	Sample: Hello World! using variables
            <P> 
	    <tscreen><verb>
	    #!/bin/bash          
	    STR="Hello World!"
	    echo $STR    
	    </verb></tscreen>

	    <P> Line 2 creates 
	    a variable called STR and assigns the string "Hello World!" to
	    it. Then the VALUE of this variable is retrieved by putting 
	    the '$' in at the beginning. Please notice (try it!) 
	    that if you don't use the '$' sign, the output of the program will
	    be different, and probably not what you want it to be.
	</sect1>
        <sect1>
	   Sample: A very simple backup script (little bit better)
	   <P> 
	   <tscreen><verb>
	   #!/bin/bash          
	   OF=/var/my-backup-$(date +%Y%m%d).tgz
	   tar -cZf $OF /home/me/
	   </verb></tscreen>
	   <P> This script introduces another thing. First 
	   of all, you should be familiarized with the variable 
	   creation and assignation on line 2. Notice the expression
	   '$(date +%Y%m%d)'.
	   If you run the script you'll notice that 
	   it runs the
	   command inside the parenthesis, capturing its output. 

	   <P> Notice that in this script, the output filename will
	   be different 
	   every day, due to the format switch to the date command(+%Y%m%d).
	   You can change this by specifying a different format.
	   <P> Some more examples:
	   <P> echo ls
	   <P> echo $(ls)
	</sect1>   
</sect>

<!-- Conditionals -->
<sect>
Conditionals
    
    <P> Conditionals let you decide whether to perform an action 
    or not, this decision is taken by evaluating an expression.

    <!-- Dry Theory -->
    <sect1>
    Dry Theory
        <P> Conditionals have many forms. The most basic form is:
	<bf/if/ <it/expression/ <bf/then/ <it/statement/
	where 'statement' is only executed if 'expression' 	
	evaluates to true.
	'2<1' is an expresion that evaluates to false, while '2>1' 
	evaluates to true.xs
	<P> Conditionals have other forms such as:
	<bf/if/ <it/expression/ 
	<bf/then/ <it/statement1/ <bf/else/ <it/statement2/.
	 Here 'statement1' is executed  if 'expression' is true,otherwise
	'statement2' is executed.
	<P> Yet another form of conditionals is:  
	<bf/if/ <it/expression1/ 
	<bf/then/ <it/statement1/ 
	<bf/else if/ <it/expression2/ <bf/then/ <it/statement2/ 
	<bf/else/ <it/statement3/.
	In this form there's added only the
	"ELSE IF 'expression2' THEN 'statement2'" which makes statement2 being
	executed if expression2 evaluates to true. The rest is as you may 
	imagine (see previous forms).
	<P> A word about syntax:
	<P> The base for the 'if' constructions in bash is this:
	<P> if [expression];
	<P> then
	<P>    code if 'expression' is true.
	<P> fi
    </sect1>
    <!-- Samples -->
        <sect1>
	Sample: Basic conditional example if .. then
            <P> 
	    <tscreen><verb>
	    #!/bin/bash
	    if [ "foo" = "foo" ]; then
	       echo expression evaluated as true
	    fi
	    </verb></tscreen>
	    <P> The code to be executed if the expression within braces 
	    is true can 
	    be found after the 'then' word and before 'fi' which indicates the end 
	    of the conditionally executed code.
	</sect1>
	<sect1>
	Sample: Basic conditional example if .. then ... else
	    <P>
	    <tscreen><verb>
	    #!/bin/bash
	    if [ "foo" = "foo" ]; then
	       echo expression evaluated as true
	    else
	       echo expression evaluated as false
	    fi
	    </verb></tscreen>
	</sect1>
	<sect1>
	Sample: Conditionals with variables
	    <P> 
	    <tscreen><verb>
	    #!/bin/bash
	    T1="foo"
	    T2="bar"
	    if [ $T1 = $T2 ]; then
	        echo expression evaluated as true
	    else
	        echo expression evaluated as false
	    fi
	    </verb></tscreen>
	</sect1>
</sect>

<!-- Loops : for, while and until -->
<sect>
Loops for, while and until
    <!-- Dry Theory -->

    <P> In this section you'll find for, while and until loops.
    <P> The <bf/for/ loop is a little bit different from other programming
    languages. Basically, it let's you iterate over a series of 
    'words' within a string.
      <P> The <bf/while/ executes a piece of code if the control expression
      is true, and only stops when it is false (or a explicit break is found
      within the executed code.
      <P> The <bf/until/ loop is almost equal to the while loop, except that
      the code is executed while the control expression evaluates to false.
      <P> If you suspect that while and until are very similar you are right.
    <!-- Samples -->    
    <sect1>
    For sample
        <P> 
	<tscreen><verb>
        #!/bin/bash
        for i in $( ls ); do
            echo item: $i
        done
        </verb></tscreen>
        <p> 
        <p> On the second line, we declare i to be the variable that
	 will take the 
        different values contained in $( ls ).
        <p> The third line could be longer if needed, or there could
	 be more lines 
        before the done (4).
        <p> 'done' (4) indicates that the code that used the value of $i has
        finished and $i can take a new value.
        <p> This script has very little sense, but a more useful way to use the
        for loop would be to use it to match only certain files on the previous
         example
    <p>
    </sect1>
    <!-- while -->
    <sect1>
    While sample
         <P> <tscreen><verb>
         #!/bin/bash 
         COUNTER=0
         while [  $COUNTER -lt 10 ]; do
             echo The counter is $COUNTER
             let COUNTER=COUNTER+1 
         done
         </verb></tscreen>
	 <P>
	 <P> This script 'emulates' the well known 
	 (C, Pascal, perl, python, etc) 'for' structure
    </sect1>
    <!-- until -->
    <sect1>
    Until sample
         <P> <tscreen><verb>
         #!/bin/bash 
         COUNTER=20
         until [  $COUNTER -lt 10 ]; do
             echo COUNTER $COUNTER
             let COUNTER-=1
         done
	 </verb></tscreen>
    </sect1>
</sect>

<!-- Functions -->
<sect>
Functions
     <P> As in almost any programming language, you can use functions to group 
     pieces of code in a more logical way or practice the divine art of recursion.
     <P> Declaring a function is just a matter of writing function my_func { my_code }.
     <P> Calling a function is just like calling another program, you just write its name.
     <P>
     <sect1>
     Functions sample
           <P> <tscreen><verb>
	   1)  #!/bin/bash 
	   2)  function quit {
	   3)      exit
	   4)  }
	   5)  function hello {
	   6)      echo Hello!
	   7)  }
	   8)  hello
	   9)  quit
	   10) echo foo 
	   </verb></tscreen>
	   <P> Lines 2-4 contain the 'quit' function. Lines 5-7 contain the 'hello' function
	   If you are not absolutely sure about what this script does, please try it!.
	   <P> Notice that a functions don't need to be declared in any specific order.
	   <P> When running the script you'll notice that first: the function 'hello' is 
	   called, second the 'quit' function, and the program never reaches line 10.
     </sect1>
</sect>


<!-- User interfaces -->
<sect>
User interfaces
     <!-- Simple menus with select -->
     <sect1>
     Using select to make simple menus
	   <P>
	   <tscreen><verb>
	   #!/bin/bash
	   OPTIONS="Hello Quit"
	   select opt in $OPTIONS; do
	       if [ $opt = "Quit" ]; then
	   	echo done
	   	exit
	       elif [ $opt = "Hello" ]; then
	   	echo Hello World
	       else
	   	clear
	   	echo bad option
	       fi
	   done
	  </verb></tscreen> 
           <P> If you run this script you'll see that it is a 
           programmer's dream for text based menus. You'll probably notice 
           that it's very similar to the 'for' construction, only rather 
           than looping for each 'word' in $OPTIONS, it prompts the user.
	   <P>
     </sect1>
    <!-- Using the command line -->
    <sect1>
    Using the command line 
         <P>
	 <tscreen><verb>
	  #!/bin/bash        
          if [ -z $1 ]; then 
              echo usage: $0 directory
              exit
          fi
          SRCD=$1
          TGTD="/var/backups/"
          OF=home-$(date +%Y%m%d).tgz
          tar -cZf $TGTD$OF $SRCD
	 </verb></tscreen>
	 <P>
	 <P> What this script does should be clear to you. The expression 
	 in the first conditional tests if the program has received an argument
	 ($1) and quits if it didn't, showing the user a little usage message.
	 The rest of the script should be clear at this point.
    </sect1>

</sect>

<!-- Misc -->
<sect>
Misc
    <!-- Reading user input  -->    
    <sect1>
    Reading user input
         <P> __TO-DO__
    </sect1>

    <!-- Arithmetic evaluation  -->    
    <sect1>
    Arithmetic evaluation
         <P> On the command line (or a shell) try this:
	 <P> echo 1 + 1
	 <P> If you expected to see '2' you'll be disappointed. What if 
	 you want BASH to evaluate some numbers you have? The solution
	 is this:
	 <P> echo $((1+1))
	 <P> This will produce a more 'logical' output. This is to evaluate an
	 arithmetic expression. You can achieve this also like this:
	 <P> echo $[1+1]
	 <P>
	 <P> If you need to use fractions, or more math or you just want it, you 
	 can use bc to evaluate arithmetic expressions.
	 <P> if i ran "echo $[3/4]" at the command prompt, it would return 0 
	 because bash  only uses integers when answering. If you  ran
	  "echo 3/4|bc -l", it would properly return 0.75.
    </sect1>

    <!-- Capturing a commands output  -->    
    <sect1>
    Capturing a commands output
         <P> __TO-DO__
    </sect1>
    
    <!-- Multiple source files -->
    <sect1>
    Multiple source files
         <P> __TO-DO__
    </sect1>
</sect>


<!-- Tables -->
<sect>
Tables
    <!-- String operators -->
    <sect1>
    String comparison operators
    
    <P> (1) s1 = s2
    <P> (2) s1 != s2
    <P> (3) s1 < s2
    <P> (4) s1 > s2
    <P> (5) -n s1 
    <P> (6) -z s1 
    <P> 
    <P> (1) s1 matches s2
    <P> (2) s1 does not match s2
    <P> (3) __TO-DO__
    <P> (4) __TO-DO__
    <P> (5) s1 is not null (contains one or more characters)
    <P> (6) s1 is null 

    </sect1>
    <!-- Arithmetic operators -->    
    <sect1>
    Arithmetic operators
    <P> +
    <P> -
    <P> *
    <P> /
    <P> % (reminder)
    </sect1>

    <!-- Arithmetic relational operators -->    
    <sect1>
    Arithmetic relational operators
    <P> -lt (<) 
    <P> -gt (>)
    <P> -le (<=)
    <P> -ge (>=)
    <P> -eq (==)
    <P> -ne (!=)
    <P> C programmer's should simple map the operator to its corresponding 
    parenthesis.
    </sect1>
    

    <!-- Useful commands  -->    
    <sect1>
    Useful commands
         <P> 
         <P> sed  (stream editor - very useful)
         <P> gawk
         <P> grep (show line matching this or not matching that)
	 <P> wc   (count words, lines)
	 <P> sort 
	 <P> bc   (more than a calculator)
	 <P> cut  (edit columns)
	 
         <P> It it higly recommended to be familiarized with
	  this programs (at least). There are tons of little 
	  programs that will let you do real magic in a 
	  command line.
	 
    </sect1>



</sect>


<!-- More Scripts -->
<sect>
More Scripts
     <sect1>
     Applying a command to all files in a directory. 
     <P> 
     </sect1>
     <sect1>
     Sample: A very simple backup script (little bit better)
	   <P>
	   <tscreen><verb>
	    #!/bin/bash          
	    SRCD="/home/"
	    TGTD="/var/backups/"
	    OF=home-$(date +%Y%m%d).tgz
	    tar -cZf $TGTD$OF $SRCD
	   </verb></tscreen> 
	   <P>
     </sect1>
     <sect1>
     File re-namer
          <P>
	  <tscreen><verb>
          
             #!/bin/sh
             # renna: rename multiple files according to several rules
             # written by felix hudson Jan - 2000
             
             #first check for the various 'modes' that this program has
             #if the first ($1) condition matches then we execute that portion of the
             #program and then exit
             
             # check for the prefix condition
             if [ $1 = p ]; then
             
             #we now get rid of the mode ($1) variable and prefix ($2)
               prefix=$2 ; shift ; shift
             
             # a quick check to see if any files were given
             # if none then its better not to do anything than rename some non-existent
             # files!!
             
               if [$1 = ]; then
                  echo "no files given"
                  exit 0
               fi
             
             # this for loop iterates through all of the files that we gave the program
             # it does one rename per file given
               for file in $*
                 do
                 mv ${file} $prefix$file
               done
             
             #we now exit the program
               exit 0
             fi
             
             # check for a suffix rename
             # the rest of this part is virtually identical to the previous section
             # please see those notes
             if [ $1 = s ]; then
               suffix=$2 ; shift ; shift
             
                if [$1 = ]; then
                 echo "no files given"
                exit 0
                fi
             
              for file in $*
               do
                mv ${file} $file$suffix
              done
             
              exit 0
             fi
             
             # check for the replacement rename
             if [ $1 = r ]; then
             
               shift
             
             # i included this bit as to not damage any files if the user does not specify
             # anything to be done
             # just a safety measure
             
               if [ $# -lt 3 ] ; then
                 echo "usage: renna r [expression] [replacement] files... "
                 exit 0
               fi
             
             # remove other information
               OLD=$1 ; NEW=$2 ; shift ; shift
             
             # this for loop iterates through all of the files that we give the program
             # it does one rename per file given using the program 'sed'
             # this is a sinple command line program that parses standard input and
             # replaces a set expression with a give string
             # here we pass it the file name ( as standard input) and replace the nessesary
             # text
             
               for file in $*
               do
                 new=`echo ${file} | sed s/${OLD}/${NEW}/g`
                 mv ${file} $new
               done
             exit 0
             fi
             
             # if we have reached here then nothing proper was passed to the program
             # so we tell the user how to use it
             echo "usage;"
             echo " renna p [prefix] files.."
             echo " renna s [suffix] files.."
             echo " renna r [expression] [replacement] files.."
             exit 0
             
             # done!
             
	  </verb></tscreen>
     </sect1>
     <sect1>
     File renamer (simple)
     <p>
     <tscreen><verb>
     #!/bin/bash
     # renames.sh
     # basic file renamer

     criteria=$1
     re_match=$2
     replace=$3
     
     for i in $( ls *$criteria* ); 
     do
         src=$i
	 tgt=$(echo $i | sed -e "s/$re_match/$replace/")
	 mv $src $tgt
     done
     </verb></tscreen>
     </sect1>
</sect>

<sect>
When something goes wrong (debugging)
     <sect1>
     Ways Calling BASH 
          <p> A nice thing to do is to add on the first line
	  <tscreen><verb>
	  #!/bin/bash -x
	  </verb></tscreen>
	  <P> This will produce some intresting output information
     </sect1>
</sect>

<sect>
About the document
     <P> Feel free to make suggestions/corrections, or whatever you think 
     it would be interesting to see in this document. I'll try to update
     it as soon as I can.
     <sect1>
     (no) warranty
          <P> This documents comes with no warranty of any kind.
	  and all that 
     </sect1>
     <sect1>
     Thanks to
     <P>
     <itemize>
         <item> Nathan Hurst for sending a lot of corrections.
	 <item> Jon Abbott for sending comments about evaluating arithmetic expressions.
	 <item> Laurent Martelli for  translating this document to French (soon here the URL)
	 <item> Felix Hudson for writing the <it/renna/ script
     </itemize>
     </sect1>
	<sect1>
	History
	<p> v5 Added the redirection section
	<p> v4 disapperd from its location due to my ex-boss and thid doc found it's new place 
	at the proper url: www.linuxdoc.org
	<p> prior:  I don't rememeber and I didn't use rcs nor cvs :(
	</sect1>

     <sect1>
     More resources
          <P>
	  <P> Introduction to bash (under BE)
	      <htmlurl url="http://org.laol.net/lamug/beforever/bashtut.htm"
		name="http://org.laol.net/lamug/beforever/bashtut.htm">
          <P> Bourne Shell Programming 
	      http://207.213.123.70/book/ 
     </sect1>
</sect>

</article>