LDP/LDP/guide/docbook/Bash-Beginners-Guide/chap1.xml

559 lines
52 KiB
XML

<chapter id="chap_01"><title>Bash and Bash scripts</title>
<abstract>
<para>In this introduction module we</para>
<para><itemizedlist>
<listitem><para>Describe some common shells</para></listitem>
<listitem><para>Point out GNU Bash advantages and features</para></listitem>
<listitem><para>Describe the shell's building blocks</para></listitem>
<listitem><para>Discuss Bash initialization files</para></listitem>
<listitem><para>See how the shell executes commands</para></listitem>
<listitem><para>Look into some simple script examples</para></listitem>
</itemizedlist></para>
</abstract>
<sect1 id="sect_01_01"><title>Common shell programs</title>
<sect2 id="sect_01_01_01"><title>General shell functions</title>
<para>The UNIX shell<indexterm><primary>shell</primary><secondary>general functions</secondary></indexterm> program interprets user commands, which are either directly entered by the user, or which can be read from a file called the shell script or shell program. Shell scripts are interpreted, not compiled. The shell reads commands from the script line per line and searches for those commands on the system (see <xref linkend="sect_01_02" />), while a compiler converts a program into machine readable form, an executable file - which may then be used in a shell script.</para>
<para>Apart from passing commands to the kernel, the main task of a shell is providing a user environment, which can be configured individually using shell resource configuration files.</para>
</sect2>
<sect2 id="sect_01_01_02"><title>Shell types</title>
<para>Just like people know different languages and dialects, your UNIX system will usually offer a variety of shell<indexterm><primary>shell</primary><secondary>types</secondary></indexterm> types:</para>
<itemizedlist>
<listitem><para><command>sh</command> or Bourne Shell: the original shell still
used on UNIX systems and in UNIX-related environments. This is the basic shell, a small program with few features. While this is not the standard shell, it is still available on every Linux system for compatibility with UNIX programs.</para></listitem>
<listitem><para><command>bash</command> or Bourne Again shell: the standard GNU
shell, intuitive and flexible. Probably most advisable for beginning users while being at the same time a powerful tool for the advanced and professional user. On Linux, <command>bash</command> is the standard shell for common users. This shell is a so-called <emphasis>superset</emphasis> of the Bourne shell, a set
of add-ons and plug-ins. This means that the Bourne Again shell is compatible with the Bourne shell: commands that work in <command>sh</command>, also work in
<command>bash</command>. However, the reverse is not always the case. All examples and exercises in this book use <command>bash</command>.</para></listitem>
<listitem><para><command>csh</command> or C shell: the syntax of this shell resembles that of the C programming language. Sometimes asked for by programmers.</para></listitem>
<listitem><para><command>tcsh</command> or TENEX C shell: a superset of the common C shell, enhancing user-friendliness and speed. That is why some also call it the Turbo C shell.</para></listitem>
<listitem><para><command>ksh</command> or the Korn shell: sometimes appreciated
by people with a UNIX background. A superset of the Bourne shell; with standard configuration a nightmare for beginning users.</para></listitem>
</itemizedlist>
<para>The file <filename>/etc/shells</filename> gives an overview of known shells on a Linux system<indexterm><primary>configuration files</primary><secondary>/etc/shells</secondary></indexterm>:</para>
<screen>
<prompt>mia:~&gt;</prompt> <command>cat <filename>/etc/shells</filename></command>
/bin/bash
/bin/sh
/bin/tcsh
/bin/csh
</screen>
<para>Your default shell is set in the <filename>/etc/passwd</filename> file<indexterm><primary>configuration files</primary><secondary>/etc/passwd</secondary></indexterm>, like this line for user <emphasis>mia</emphasis>:</para>
<screen>
mia:L2NOfqdlPrHwE:504:504:Mia Maya:/home/mia:/bin/bash
</screen>
<para>To switch<indexterm><primary>shell</primary><secondary>switch between shells</secondary></indexterm> from one shell to another, just enter the name of the new shell
in the active terminal. The system finds the directory where the name occurs using the <varname>PATH</varname> settings, and since a shell is an executable file (program), the current shell activates it and it gets executed. A new prompt is usually shown, because each shell has its typical appearance:</para>
<screen>
<prompt>mia:~&gt;</prompt> <command>tcsh</command>
[mia@post21 ~]$
</screen>
</sect2>
</sect1>
<sect1 id="sect_01_02"><title>Advantages of the Bourne Again SHell</title>
<sect2 id="sect_01_02_01"><title>Bash is the GNU shell</title>
<para>The GNU project (GNU's Not UNIX) provides tools for UNIX-like system administration which are free software and comply to UNIX standards.</para>
<para>Bash<indexterm><primary>Bash</primary><secondary>advantages</secondary></indexterm> is an sh-compatible shell that incorporates useful features from the Korn shell (ksh) and C shell (csh). It is intended to conform to the IEEE POSIX P1003.2/ISO 9945.2 Shell and Tools standard. It offers functional improvements over sh for both programming and interactive use; these include command line editing, unlimited size command history, job control, shell functions and aliases, indexed arrays of unlimited size, and integer arithmetic in any base from two to sixty-four. Bash can run most sh scripts without modification.</para>
<para>Like the other GNU projects, the bash initiative was started to preserve, protect and promote the freedom to use, study, copy, modify and redistribute software. It is generally known that such conditions stimulate creativity. This was also the case with the bash program, which has a lot of extra features that other shells can't offer.</para>
</sect2>
<sect2 id="sect_01_02_02"><title>Features only found in bash</title>
<sect3 id="sect_01_02_02_01"><title>Invocation</title>
<para>In addition<indexterm><primary>Bash</primary><secondary>features</secondary></indexterm> to the single-character shell command line options which can generally be configured using the <command>set</command> shell built-in command, there are several multi-character options that you can use. We will come across a couple of the more popular options<indexterm><primary>features</primary><secondary>invocation</secondary></indexterm> in this and the following chapters; the complete list can be found in the Bash info pages, <menuchoice><guimenu>Bash features</guimenu><guimenuitem>Invoking Bash</guimenuitem></menuchoice>.</para>
</sect3>
<sect3 id="sect_01_02_02_02"><title>Bash startup files</title>
<para>Startup<indexterm><primary>Bash</primary><secondary>startup files</secondary></indexterm> files are scripts that are read and executed by Bash when it starts. The following subsections<indexterm><primary>features</primary><secondary>startup files</secondary></indexterm> describe different ways to start the shell, and the startup files that are read consequently.</para>
<sect4 id="sect_01_02_02_02_01"><title>Invoked as an interactive login shell, or with `--login'</title>
<para>Interactive<indexterm><primary>invocation</primary><secondary>interactive login shell</secondary></indexterm> means you can enter commands. The shell is not running because a script has been activated. A login shell means that you got the shell after authenticating to the system, usually by giving your user name and password.</para>
<para>Files read:</para>
<itemizedlist>
<listitem><para><filename>/etc/profile</filename></para></listitem>
<listitem><para><filename>~/.bash_profile</filename>, <filename>~/.bash_login</filename> or <filename>~/.profile</filename>: first existing readable file is read</para></listitem>
<listitem><para><filename>~/.bash_logout</filename> upon logout.</para></listitem>
</itemizedlist>
<para>Error messages are printed if configuration files exist but are not readable. If a file does not exist, bash searches for the next.</para>
</sect4>
<sect4 id="sect_01_02_02_02_02"><title>Invoked as an interactive non-login shell</title>
<para>A non-login<indexterm><primary>invocation</primary><secondary>interactive non-login shell</secondary></indexterm> shell means that you did not have to authenticate to the system. For instance, when you open a terminal using an icon, or a menu item, that is a non-login shell.</para>
<para>Files read:</para>
<itemizedlist>
<listitem><para><filename>~/.bashrc</filename></para></listitem>
</itemizedlist>
<para>This file is usually referred to in <filename>~/.bash_profile</filename>:</para>
<cmdsynopsis><command>if <function>[ -f <filename>~/.bashrc</filename> ]</function>; then . <filename>~/.bashrc</filename>; fi</command></cmdsynopsis>
<para>See <xref linkend="chap_07" /> for more information on the <command>if</command> construct.</para>
</sect4>
<sect4 id="sect_01_02_02_02_03"><title>Invoked non-interactively</title>
<para>All scripts use non-interactive<indexterm><primary>invocation</primary><secondary>non-interactive</secondary></indexterm> shells. They are programmed to do certain tasks and cannot be instructed to do other jobs than those for which they are programmed.</para>
<para>Files read:</para>
<itemizedlist>
<listitem><para>defined<indexterm><primary>variables</primary><secondary>BASH_ENV</secondary></indexterm> by <varname>BASH_ENV</varname></para></listitem>
</itemizedlist>
<para><varname>PATH</varname> is not used to search for this file, so if you want to use it, best refer to it by giving the full path and file name.</para>
</sect4>
<sect4 id="sect_01_02_02_02_04"><title>Invoked with the sh command</title>
<para>Bash<indexterm><primary>invocation</primary><secondary>invoked as sh</secondary></indexterm> tries to behave as the historical Bourne <command>sh</command> program while conforming to the POSIX standard as well.</para>
<para>Files read:</para>
<itemizedlist>
<listitem><para><filename>/etc/profile</filename></para></listitem>
<listitem><para><filename>~/.profile</filename></para></listitem>
</itemizedlist>
<para>When invoked interactively, the <varname>ENV</varname> variable<indexterm><primary>variables</primary><secondary>ENV</secondary></indexterm> can point to extra startup information.</para>
</sect4>
<sect4 id="sect_01_02_02_02_05"><title>POSIX mode</title>
<para>This option<indexterm><primary>invocation</primary><secondary>POSIX mode</secondary></indexterm> is enabled either using the <command>set</command> built-in:</para>
<cmdsynopsis><command>set <option>-o posix</option></command></cmdsynopsis>
<para>or by calling the <command>bash</command> program with the <option>--posix</option> option. Bash will then try to behave as compliant as possible to the POSIX standard for shells. Setting the <varname>POSIXLY_CORRECT</varname> variable<indexterm><primary>variables</primary><secondary>POSIXLY_CORRECT</secondary></indexterm> does the same.</para>
<para>Files read:</para>
<itemizedlist>
<listitem><para>defined by <varname>ENV</varname> variable.</para></listitem>
</itemizedlist>
</sect4>
<sect4 id="sect_01_02_02_02_06"><title>Invoked remotely</title>
<para>Files read when invoked<indexterm><primary>invocation</primary><secondary>remote invocation</secondary></indexterm> by <command>rshd</command>:</para>
<itemizedlist>
<listitem><para><filename>~/.bashrc</filename></para></listitem>
</itemizedlist>
<warning><title>Avoid use of r-tools</title>
<para>Be aware of the dangers when using tools such as <command>rlogin</command>, <command>telnet</command>, <command>rsh</command> and <command>rcp</command>. They are intrinsically insecure because confidential data is sent over the network unencrypted. If you need tools for remote execution, file transfer and so on, use an implementation of Secure SHell, generally known as SSH, freely available from <ulink url="http://www.openssh.org">http://www.openssh.org</ulink>. Different client programs are available for non-UNIX systems as well, see your local software mirror.</para>
</warning>
</sect4>
<sect4 id="sect_01_02_02_02_07"><title>Invoked when UID is not equal to EUID</title>
<para>No startup<indexterm><primary>invocation</primary><secondary>UID &lt;&gt; EUID</secondary></indexterm> files are read in this case.</para>
</sect4>
</sect3>
<sect3 id="sect_01_02_02_03"><title>Interactive shells</title>
<sect4 id="sect_01_02_02_03_01"><title>What is an interactive shell?</title>
<para>An interactive<indexterm><primary>shell</primary><secondary>interactive</secondary></indexterm> shell generally reads from, and writes to, a user's terminal: input and output are connected to a terminal. Bash interactive behavior<indexterm><primary>features</primary><secondary>interactive shells</secondary></indexterm> is started when the <command>bash</command> command is called upon without non-option arguments, except when the option is a string to read from or when the shell is invoked to read from standard input, which allows for positional parameters to be set (see <xref linkend="chap_03" /> ).</para>
</sect4>
<sect4 id="sect_01_02_02_03_02"><title>Is this shell interactive?</title>
<para>Test by looking at the content of the special parameter <varname>-</varname>, it contains an 'i' when the shell is interactive:</para>
<screen>
<prompt>eddy:~&gt;</prompt> <command>echo <varname>$-</varname></command>
himBH
</screen>
<para>In non-interactive shells, the prompt, <varname>PS1</varname>, is unset.</para>
</sect4>
<sect4 id="sect_01_02_02_03_03"><title>Interactive shell behavior</title>
<para>Differences in interactive<indexterm><primary>interactive shell</primary><secondary>behavior</secondary></indexterm> mode:</para>
<itemizedlist>
<listitem><para>Bash reads startup files.</para></listitem>
<listitem><para>Job control enabled by default.</para></listitem>
<listitem><para>Prompts are set, <varname>PS2</varname> is enabled for multi-line<indexterm><primary>variables</primary><secondary>PS2</secondary></indexterm> commands, it is usually set to <quote>&gt;</quote>. This is also the prompt you get when the shell thinks you entered an unfinished command, for instance when you forget quotes, command structures that cannot be left out, etc.</para></listitem>
<listitem><para>Commands are by default read from the command line using <command>readline</command>.</para></listitem>
<listitem><para>Bash interprets the shell option <option>ignoreeof<indexterm><primary>options</primary><secondary>ignoreeof</secondary></indexterm></option> instead of exiting immediately upon receiving EOF (End Of File).</para></listitem>
<listitem><para>Command history and history expansion are enabled by default. History is saved in the file pointed to by<indexterm><primary>variables</primary><secondary>HISTFILE</secondary></indexterm> <varname>HISTFILE</varname> when the shell exits. By default, <varname>HISTFILE</varname> points to <filename>~/.bash_history</filename>.</para></listitem>
<listitem><para>Alias expansion is enabled.</para></listitem>
<listitem><para>In the absence of traps, the <option>SIGTERM</option> signal is ignored.</para></listitem>
<listitem><para>In the absence of traps, <option>SIGINT</option> is caught and handled. Thus, typing <keycap>Ctrl</keycap>+<keycap>C</keycap>, for example, will not quit your interactive shell.</para></listitem>
<listitem><para>Sending <option>SIGHUP</option> signals to all jobs on exit is configured with the <option>huponexit</option> option.</para></listitem>
<listitem><para>Commands are executed upon read.</para></listitem>
<listitem><para>Bash checks for mail periodically.</para></listitem>
<listitem><para>Bash can be configured to exit when it encounters unreferenced variables. In interactive mode this behavior is disabled.</para></listitem>
<listitem><para>When shell built-in commands encounter redirection errors, this will not cause the shell to exit.</para></listitem>
<listitem><para>Special built-ins returning errors when used in POSIX mode don't cause the shell to exit. The built-in commands are listed in <xref linkend="sect_01_03_02" />.</para></listitem>
<listitem><para>Failure of <command>exec</command> will not exit the shell.</para></listitem>
<listitem><para>Parser syntax errors don't cause the shell to exit.</para></listitem>
<listitem><para>Simple spell check for the arguments to the <command>cd</command> built-in is enabled by default.</para></listitem>
<listitem><para>Automatic exit after the length of time specified in the <varname>TMOUT</varname> variable has passed, is enabled.</para></listitem>
</itemizedlist>
<para>More information:</para>
<itemizedlist>
<listitem><para><xref linkend="sect_03_02" /></para></listitem>
<listitem><para><xref linkend="sect_03_06" /></para></listitem>
<listitem><para>See <xref linkend="chap_12" /> for more about signals.</para></listitem>
<listitem><para><xref linkend="sect_03_04" /> discusses the various expansions performed upon entering a command.</para></listitem>
</itemizedlist>
</sect4>
</sect3>
<sect3 id="sect_01_02_02_04"><title>Conditionals</title>
<para>Conditional<indexterm><primary>features</primary><secondary>conditionals</secondary></indexterm> expressions are used by the <command>[[</command> compound command and by the <command>test</command> and <command>[</command> built-in commands.</para>
<para>Expressions may be unary or binary. Unary expressions are often used to examine the status of a file. You only need one object, for instance a file, to do the operation on.</para>
<para>There are string operators and numeric comparison operators as well; these are binary operators, requiring two objects to do the operation on. If the <option>FILE</option> argument to one of the primaries is in the form <filename>/dev/fd/N</filename>, then file descriptor N is checked. If the <option>FILE</option> argument to one of the primaries is one of <filename>/dev/stdin</filename>, <filename>/dev/stdout</filename> or <filename>/dev/stderr</filename>, then file descriptor 0, 1 or 2 respectively is checked.</para>
<para>Conditionals are discussed in detail in <xref linkend="chap_07" />.</para>
<para>More information about the file descriptors in <xref linkend="sect_08_02_03" />.</para>
</sect3>
<sect3 id="sect_01_02_02_05"><title>Shell arithmetic</title>
<para>The shell allows arithmetic<indexterm><primary>features</primary><secondary>shell arithmetic</secondary></indexterm> expressions to be evaluated, as one of
the shell expansions or by the <command>let</command> built-in.</para>
<para>Evaluation is done in fixed-width integers with no check for overflow, though division by 0 is trapped and flagged as an error. The operators and their precedence and associativity are the same as in the C language, see <xref linkend="chap_03" />.</para>
</sect3>
<sect3 id="sect_01_02_02_06"><title>Aliases</title>
<para>Aliases<indexterm><primary>features</primary><secondary>aliases</secondary></indexterm> allow a string to be substituted for a word when it is used as the first word of a simple command. The shell maintains a list of aliases that may be set and unset with the <command>alias</command> and <command>unalias</command> commands.</para>
<para>Bash always reads at least one complete line of input before executing any of the commands on that line. Aliases are expanded when a command is read, not when it is executed. Therefore, an alias definition appearing on the same line as another command does not take effect until the next line of input is read. The commands following the alias definition on that line are not affected by the new alias.</para>
<para>Aliases are expanded when a function definition is read, not when the function is executed, because a function definition is itself a compound command. As a consequence, aliases defined in a function are not available until after that function is executed.</para>
<para>We will discuss aliases in detail in <xref linkend="sect_03_05" />.</para>
</sect3>
<sect3 id="sect_01_02_02_07"><title>Arrays</title>
<para>Bash provides one-dimensional array<indexterm><primary>features</primary><secondary>arrays</secondary></indexterm> variables. Any variable may be used as an array; the <command>declare</command> built-in will explicitly declare an array. There is no maximum limit on the size of an array, nor any requirement that members be indexed or assigned contiguously. Arrays are zero-based. See <xref linkend="chap_10" />.</para>
</sect3>
<sect3 id="sect_01_02_02_08"><title>Directory stack</title>
<para>The directory stack<indexterm><primary>features</primary><secondary>directory stack</secondary></indexterm> is a list of recently-visited directories. The <command>pushd</command> built-in<indexterm><primary>built-ins</primary><secondary>pushd</secondary></indexterm> adds directories to the stack as it changes the current directory, and the <command>popd</command> built-in<indexterm><primary>built-in</primary><secondary>popd</secondary></indexterm> removes specified directories from the stack and changes the current directory to the directory removed.</para>
<para>Content can be displayed issuing the <command>dirs</command> command or by checking the content of the <varname>DIRSTACK<indexterm><primary>variables</primary><secondary>DIRSTACK</secondary></indexterm></varname> variable.</para>
<para>More information about the workings of this mechanism can be found in the Bash info pages.</para>
</sect3>
<sect3 id="sect_01_02_02_09"><title>The prompt</title>
<para>Bash makes playing with the prompt<indexterm><primary>features</primary><secondary>prompt</secondary></indexterm> even more fun. See the section <emphasis>Controlling the Prompt</emphasis> in the Bash info pages.</para>
</sect3>
<sect3 id="sect_01_02_02_10"><title>The restricted shell</title>
<para>When invoked as <command>rbash<indexterm><primary>features</primary><secondary>restricted shell</secondary></indexterm></command> or with the <option>--restricted<indexterm><primary>options</primary><secondary>restricted</secondary></indexterm></option> or <option>-r</option> option, the following happens<indexterm><primary>restricted shell</primary><secondary>behavior</secondary></indexterm>:</para>
<itemizedlist>
<listitem><para>The <command>cd</command> built-in is disabled.</para></listitem>
<listitem><para>Setting or unsetting <varname>SHELL<indexterm><primary>variables</primary><secondary>SHELL</secondary></indexterm></varname>, <varname>PATH<indexterm><primary>variables</primary><secondary>PATH</secondary></indexterm></varname>, <varname>ENV<indexterm><primary>variables</primary><secondary>ENV</secondary></indexterm></varname> or <varname>BASH_ENV<indexterm><primary>variables</primary><secondary>BASH_ENV</secondary></indexterm></varname> is not possible.</para></listitem>
<listitem><para>Command names can no longer contain slashes.</para></listitem>
<listitem><para>Filenames containing a slash are not allowed with the <command>.</command> (<command>source</command>) built-in<indexterm><primary>built-ins</primary><secondary>source</secondary></indexterm> command.</para></listitem>
<listitem><para>The <command>hash</command> built-in<indexterm><primary>built-ins</primary><secondary>hash</secondary></indexterm> does not accept slashes with the <option>-p</option> option.</para></listitem>
<listitem><para>Import of functions<indexterm><primary>functions</primary><secondary>restricted shell</secondary></indexterm> at startup is disabled.</para></listitem>
<listitem><para><varname>SHELLOPTS<indexterm><primary>variables</primary><secondary>SHELLOPTS</secondary></indexterm></varname> is ignored at startup.</para></listitem>
<listitem><para>Output redirection using <command>&gt;</command>, <command>&gt;|</command>, <command>&gt;&lt;</command>, <command>&gt;&amp;</command>, <command>&amp;&gt;</command> and <command>&gt;&gt;</command> is disabled.</para></listitem>
<listitem><para>The <command>exec</command> built-in is disabled.</para></listitem>
<listitem><para>The <option>-f</option> and <option>-d</option> options are disabled for the <command>enable</command> built-in<indexterm><primary>built-ins</primary><secondary>enable</secondary></indexterm>.</para></listitem>
<listitem><para>A default <varname>PATH</varname> cannot be specified with the <command>command</command> built-in.</para></listitem>
<listitem><para>Turning off restricted mode is not possible.</para></listitem>
</itemizedlist>
<para>When a command that is found to be a shell script is executed, <command>rbash</command> turns off any restrictions in the shell spawned to execute the script.</para>
<para>More information:</para>
<itemizedlist>
<listitem><para><xref linkend="sect_03_02" /></para></listitem>
<listitem><para><xref linkend="sect_03_06" /></para></listitem>
<listitem><para><menuchoice><guimenu>Info Bash</guimenu><guisubmenu>Basic Shell Features</guisubmenu><guimenuitem>Redirections</guimenuitem></menuchoice></para></listitem>
<listitem><para><xref linkend="sect_08_02_03" />: advanced redirection</para></listitem>
</itemizedlist>
</sect3>
</sect2>
</sect1>
<sect1 id="sect_01_03"><title>Executing commands</title>
<sect2 id="sect_01_03_01"><title>General</title>
<para>Bash determines the type of program that is to be execute<indexterm><primary>commands</primary><secondary>execution</secondary></indexterm>d. Normal programs are system commands that exist in compiled form on your system. When such a program is executed, a new process is created because Bash makes an exact copy of itself. This child process has the same environment as its parent, only the process ID number is different. This procedure is called <emphasis>forking</emphasis>.</para>
<para>After the forking process, the address space of the child process is overwritten with the new process data. This is done through an <emphasis>exec</emphasis> call to the system.</para>
<para>The <emphasis>fork-and-exec</emphasis> mechanism<indexterm><primary>commands</primary><secondary>fork-and-exec</secondary></indexterm> thus switches an old command with a new, while the environment in which the new program is executed remains the same, including configuration of input and output devices, environment variables and priority. This mechanism is used to create all UNIX processes, so it also applies to the Linux operating system. Even the first process, <command>init</command>, with process ID 1, is forked during the boot procedure in the so-called <emphasis>bootstrapping</emphasis> procedure.</para>
</sect2>
<sect2 id="sect_01_03_02"><title>Shell built-in commands</title>
<para>Built-in commands<indexterm><primary>commands</primary><secondary>built-in commands</secondary></indexterm> are contained within the shell itself. When the name of a built-in command is used as the first word of a simple command, the shell executes the command directly, without creating a new process. Built-in commands are necessary to implement functionality impossible or inconvenient to obtain with separate utilities.</para>
<para>Bash supports 3 types of built-in commands:</para>
<itemizedlist>
<listitem><para>Bourne<indexterm><primary>built-ins</primary><secondary>Bourne Shell built-ins</secondary></indexterm> Shell built-ins:</para>
<para><command>:</command>, <command>.</command>, <command>break</command>, <command>cd</command>, <command>continue</command>, <command>eval</command>, <command>exec</command>, <command>exit</command>, <command>export</command>, <command>getopts</command>, <command>hash</command>, <command>pwd</command>, <command>readonly</command>, <command>return</command>, <command>set</command>, <command>shift</command>, <command>test</command>, <command>[</command>, <command>times</command>, <command>trap</command>, <command>umask</command> and <command>unset</command>.</para>
</listitem>
<listitem><para>Bash<indexterm><primary>built-ins</primary><secondary>Bash built-ins</secondary></indexterm> built-in commands:</para>
<para><command>alias</command>, <command>bind</command>, <command>builtin</command>, <command>command</command>, <command>declare</command>, <command>echo</command>, <command>enable</command>, <command>help</command>, <command>let</command>, <command>local</command>, <command>logout</command>, <command>printf</command>, <command>read</command>, <command>shopt</command>, <command>type</command>, <command>typeset</command>, <command>ulimit</command> and <command>unalias</command>.</para>
</listitem>
<listitem><para>Special<indexterm><primary>built-ins</primary><secondary>special built-ins</secondary></indexterm> built-in commands:</para>
<para>When Bash is executing in POSIX mode, the special built-ins differ from other built-in commands in three respects:</para>
<orderedlist>
<listitem><para>Special built-ins are found before shell functions during command lookup.</para></listitem>
<listitem><para>If a special built-in returns an error status, a non-interactive shell exits.</para></listitem>
<listitem><para>Assignment statements preceding the command stay in effect in the shell environment after the command completes.</para></listitem>
</orderedlist>
<para>The POSIX special built-ins are <command>:</command>, <command>.</command>, <command>break</command>, <command>continue</command>, <command>eval</command>, <command>exec</command>, <command>exit</command>, <command>export</command>, <command>readonly</command>, <command>return</command>, <command>set</command>, <command>shift</command>, <command>trap</command> and <command>unset</command>.</para>
</listitem>
</itemizedlist>
<para>Most of these built-ins will be discussed in the next chapters. For those commands for which this is not the case, we refer to the Info pages.</para>
</sect2>
<sect2 id="sect_01_03_03"><title>Executing programs from a script</title>
<para>When the program<indexterm><primary>commands</primary><secondary>script execution</secondary></indexterm> being executed is a shell script, bash will create a new bash process using a <emphasis>fork</emphasis>. This subshell reads the lines from the shell script one line at a time. Commands on each line are read, interpreted and executed as if they would have come directly from the keyboard.</para>
<para>While the subshell processes each line of the script, the parent shell waits for its child process to finish. When there are no more lines in the shell script to read, the subshell terminates. The parent shell awakes and displays a new prompt.</para>
</sect2>
</sect1>
<sect1 id="sect_01_04"><title>Building blocks</title>
<sect2 id="sect_01_04_01"><title>Shell building blocks</title>
<sect3 id="sect_01_04_01_01"><title>Shell syntax</title>
<para>If input is not commented, the shell reads<indexterm><primary>shell</primary><secondary>syntax</secondary></indexterm> it and divides it into words and operators, employing quoting rules to define the meaning of each character of input. Then these words and operators are translated into commands and other constructs, which return an exit status available for inspection or processing. The above fork-and-exec scheme is only applied after the shell has analyzed input in the following<indexterm><primary>input</primary><secondary>analysis</secondary></indexterm> way:</para>
<itemizedlist>
<listitem><para>The shell reads its input from a file, from a string or from the user's terminal.</para></listitem>
<listitem><para>Input is broken up into words and operators, obeying the quoting rules, see <xref linkend="chap_03" />. These tokens are separated by <emphasis>metacharacters</emphasis>. Alias expansion is performed.</para></listitem>
<listitem><para>The shell <emphasis>parses</emphasis> (analyzes and substitutes) the tokens into simple and compound commands.</para></listitem>
<listitem><para>Bash performs various shell expansions, breaking the expanded tokens into lists of filenames and commands and arguments.</para></listitem>
<listitem><para>Redirection is performed if necessary, redirection operators and their operands are removed from the argument list.</para></listitem>
<listitem><para>Commands are executed.</para></listitem>
<listitem><para>Optionally the shell waits for the command to complete and collects its exit status.</para></listitem>
</itemizedlist>
</sect3>
<sect3 id="sect_01_04_01_02"><title>Shell commands</title>
<para>A simple shell command such as <command>touch <filename>file1</filename> <filename>file2</filename> <filename>file3</filename></command> consists of the command itself followed by arguments<indexterm><primary>arguments</primary><secondary>to a command</secondary></indexterm>, separated by spaces.</para>
<para>More complex shell commands are composed of simple commands arranged together in a variety of ways: in a pipeline in which the output of one command becomes the input of a second, in a loop or conditional construct, or in some other grouping. A couple of examples:</para>
<cmdsynopsis><command>ls | more</command></cmdsynopsis>
<cmdsynopsis><command>gunzip <filename>file.tar.gz</filename> | tar <option>xvf</option> <parameter>-</parameter></command></cmdsynopsis>
</sect3>
<sect3 id="sect_01_04_01_03"><title>Shell functions</title>
<para>Shell functions<indexterm><primary>functions</primary><secondary>execution</secondary></indexterm> are a way to group commands for later execution using a single name for the group. They are executed just like a <quote>regular</quote> command. When the name of a shell function is used as a simple command name, the list of commands associated with that function name is executed.</para>
<para>Shell functions are executed in the current shell context; no new process is created to interpret them.</para>
<para>Functions are explained in <xref linkend="chap_11" />.</para>
</sect3>
<sect3 id="sect_01_04_01_04"><title>Shell parameters</title>
<para>A parameter<indexterm><primary>shell</primary><secondary>parameters</secondary></indexterm> is an entity that stores values. It can be a name, a number or a special value. For the shell's purpose, a variable<indexterm><primary>variables</primary><secondary>definition</secondary></indexterm> is a parameter that stores a name. A variable has a value and zero or more attributes. Variables are created with the <command>declare<indexterm><primary>built-ins</primary><secondary>declare</secondary></indexterm></command> shell built-in command.</para>
<para>If no value is given, a variable is assigned the null string. Variables can only be removed with the <command>unset</command> built-in.</para>
<para>Assigning variables is discussed in <xref linkend="sect_03_02" />, advanced use of variables in <xref linkend="chap_10" />.</para>
</sect3>
<sect3 id="sect_01_04_01_05"><title>Shell expansions</title>
<para>Shell expansion<indexterm><primary>shell</primary><secondary>expansion</secondary></indexterm> is performed after each command line has been split into tokens. These are the expansions performed:</para>
<itemizedlist>
<listitem><para>Brace expansion</para></listitem>
<listitem><para>Tilde expansion</para></listitem>
<listitem><para>Parameter and variable expansion</para></listitem>
<listitem><para>Command substitution</para></listitem>
<listitem><para>Arithmetic expansion</para></listitem>
<listitem><para>Word splitting</para></listitem>
<listitem><para>Filename expansion</para></listitem>
</itemizedlist>
<para>We'll discuss these expansion types in detail in <xref linkend="sect_03_04" />.</para>
</sect3>
<sect3 id="sect_01_04_01_06"><title>Redirections</title>
<para>Before a command is executed, its input and output may be redirected<indexterm><primary>shell</primary><secondary>redirections</secondary></indexterm> using a special notation interpreted by the shell. Redirection may also be used to open and close files for the current shell execution environment.</para>
</sect3>
<sect3 id="sect_01_04_01_07"><title>Executing commands</title>
<para>When executing a command, the words that the parser has marked as variable assignments (preceding the command name) and redirections are saved for later reference. Words that are not variable assignments or redirections are expanded; the first remaining word after expansion is taken to be the name of the command and the rest are arguments<indexterm><primary>arguments</primary><secondary>definition</secondary></indexterm> to that command. Then redirections are performed, then strings assigned to variables are expanded. If no command name results, variables will affect the current shell environment.</para>
<para>An important part of the tasks of the shell is to search<indexterm><primary>commands</primary><secondary>search for commands</secondary></indexterm> for commands. Bash does this as follows:</para>
<itemizedlist>
<listitem><para>Check whether the command contains slashes. If not, first check with the function list to see if it contains a command by the name we are looking for.</para></listitem>
<listitem><para>If command is not a function, check for it in the built-in list.</para></listitem>
<listitem><para>If command is neither a function nor a built-in, look for it analyzing the directories listed in <varname>PATH<indexterm><primary>variables</primary><secondary>PATH</secondary></indexterm></varname>. Bash uses a <emphasis>hash table</emphasis> (data storage area in memory) to remember the full path names of executables so extensive <varname>PATH</varname> searches can be avoided.</para></listitem>
<listitem><para>If the search is unsuccessful, bash prints an error message and returns an exit status of 127.</para></listitem>
<listitem><para>If the search was successful or if the command contains slashes, the shell executes the command in a separate execution environment.</para></listitem>
<listitem><para>If execution fails because the file is not executable and not a directory, it is assumed to be a shell script.</para></listitem>
<listitem><para>If the command was not begun asynchronously, the shell waits for the command to complete and collects its exit status.</para></listitem>
</itemizedlist>
</sect3>
<sect3 id="sect_01_04_01_08"><title>Shell scripts</title>
<para>When a file containing<indexterm><primary>features</primary><secondary>scripts</secondary></indexterm> shell commands is used as the first non-option argument when invoking Bash (without <option>-c</option> or <option>-s</option>, this will create a non-interactive shell. This shell first searches for the script file in the current directory, then looks in <varname>PATH</varname> if the file cannot be found there.</para>
</sect3>
</sect2>
</sect1>
<sect1 id="sect_01_05"><title>Developing good scripts</title>
<sect2 id="sect_01_05_01"><title>Properties of good scripts</title>
<para>This guide is mainly about the last shell building block, scripts. Some general considerations<indexterm><primary>scripts</primary><secondary>considerations</secondary></indexterm> before we continue:</para>
<orderedlist>
<listitem><para>A script should run without errors.</para></listitem>
<listitem><para>It should perform the task for which it is intended.</para></listitem>
<listitem><para>Program logic is clearly defined and apparent.</para></listitem>
<listitem><para>A script does not do unnecessary work.</para></listitem>
<listitem><para>Scripts should be reusable.</para></listitem>
</orderedlist>
</sect2>
<sect2 id="sect_01_05_02"><title>Structure</title>
<para>The structure<indexterm><primary>scripts</primary><secondary>structure</secondary></indexterm> of a shell script is very flexible. Even though in Bash a lot of freedom is granted, you must ensure correct logic, flow control and efficiency so that users executing the script can do so easily and correctly.</para>
<para>When starting on a new script, ask yourself the following questions:</para>
<itemizedlist>
<listitem><para>Will I be needing any information from the user or from the user's environment?</para></listitem>
<listitem><para>How will I store that information?</para></listitem>
<listitem><para>Are there any files that need to be created? Where and with which permissions and ownerships?</para></listitem>
<listitem><para>What commands will I use? When using the script on different systems, do all these systems have these commands in the required versions?</para></listitem>
<listitem><para>Does the user need any notifications? When and why?</para></listitem>
</itemizedlist>
</sect2>
<sect2 id="sect_01_05_03"><title>Terminology</title>
<para>The table below gives an overview of programming terms<indexterm><primary>scripts</primary><secondary>terminology</secondary></indexterm> that you need to be familiar with:</para>
<table id="table_01_01" frame="all">
<title>Overview of programming terms</title>
<tgroup cols="2" align="left" colsep="1" rowsep="1">
<thead>
<row><entry>Term</entry><entry>What is it?</entry></row>
</thead>
<tbody>
<row><entry>Command control</entry><entry>Testing exit status of a command in order to determine whether a portion of the program should be executed.</entry></row>
<row><entry>Conditional branch</entry><entry>Logical point in the program when a condition determines what happens next.</entry></row>
<row><entry>Logic flow</entry><entry>The overall design of the program. Determines logical sequence of tasks so that the result is successful and controlled.</entry></row>
<row><entry>Loop</entry><entry>Part of the program that is performed zero or more times.</entry></row>
<row><entry>User input</entry><entry>Information provided by an external source while the program is running, can be stored and recalled when needed.</entry></row>
</tbody>
</tgroup>
</table>
</sect2>
<sect2 id="sect_01_05_04"><title>A word on order and logic</title>
<para>In order to speed up the developing process, the logical<indexterm><primary>scripts</primary><secondary>logic</secondary></indexterm> order of a program should be thought over in advance. This is your first step when developing a script.</para>
<para>A number of methods can be used; one of the most common is working with lists. Itemizing the list of tasks involved in a program allows you to describe each process. Individual tasks can be referenced by their item number.</para>
<para>Using your own spoken language to pin down the tasks to be executed by your program will help you to create an understandable form of your program. Later, you can replace the everyday language statements with shell language words and constructs.</para>
<para>The example<indexterm><primary>scripts</primary><secondary>logic flow example</secondary></indexterm> below shows such a logic flow design. It describes the rotation of log files. This example shows a possible repetitive loop, controlled by the number of base log files you want to rotate:</para>
<orderedlist>
<listitem><para>Do you want to rotate logs?</para>
<orderedlist>
<listitem><para>If yes:</para>
<orderedlist>
<listitem><para>Enter directory name containing the logs to be rotated.</para></listitem>
<listitem><para>Enter base name of the log file.</para></listitem>
<listitem><para>Enter number of days logs should be kept.</para></listitem>
<listitem><para>Make settings permanent in user's crontab file.</para></listitem>
</orderedlist></listitem>
<listitem><para>If no, go to step 3.</para></listitem>
</orderedlist></listitem>
<listitem><para>Do you want to rotate another set of logs?</para>
<orderedlist>
<listitem><para>If yes: repeat step 1.</para></listitem>
<listitem><para>If no: go to step 3.</para></listitem>
</orderedlist></listitem>
<listitem><para>Exit</para></listitem>
</orderedlist>
<para>The user should provide information for the program to do something. Input from the user must be obtained and stored. The user should be notified that his crontab will change.</para>
</sect2>
<sect2 id="sect_01_05_05"><title>An example<indexterm><primary>scripts</primary><secondary>example</secondary></indexterm> Bash script: mysystem.sh</title>
<para>The <filename>mysystem.sh</filename> script below executes some well-known commands (<command>date</command>, <command>w</command>, <command>uname</command>, <command>uptime</command>) to display information about you and your machine.</para>
<screen>
<prompt>tom:~&gt;</prompt> <command>cat <option>-n</option> <filename>mysystem.sh</filename></command>
1 #!/bin/bash
2 clear
3 echo "This is information provided by mysystem.sh. Program starts now."
4
5 echo "Hello, $USER"
6 echo
7
8 echo "Today's date is `date`, this is week `date +"%V"`."
9 echo
10
11 echo "These users are currently connected:"
12 w | cut -d " " -f 1 - | grep -v USER | sort -u
13 echo
14
15 echo "This is `uname -s` running on a `uname -m` processor."
16 echo
17
18 echo "This is the uptime information:"
19 uptime
20 echo
21
22 echo "That's all folks!"
</screen>
<para>A script always starts with the same two characters, <quote>#!</quote>. After that, the shell that will execute the commands following the first line is defined. This script starts with clearing the screen on line 2. Line 3 makes it print a message, informing the user about what is going to happen. Line 5 greets the user. Lines 6, 9, 13, 16 and 20 are only there for orderly output display purposes. Line 8 prints the current date and the number of the week. Line 11 is again an informative message, like lines 3, 18 and 22. Line 12 formats the output of the <command>w</command>; line 15 shows operating system and CPU information. Line 19 gives the uptime and load information.</para>
<para>Both <command>echo</command> and <command>printf</command> are Bash built-in commands. The first always exits with a 0 status, and simply prints arguments followed by an end of line character on the standard output, while the latter allows for definition of a formatting string and gives a non-zero exit status code upon failure.</para>
<para>This is the same script using the <command>printf</command> built-in:</para>
<screen>
<prompt>tom:~&gt;</prompt> <command>cat <filename>mysystem.sh</filename></command>
#!/bin/bash
clear
printf "This is information provided by mysystem.sh. Program starts now.\n"
printf "Hello, $USER.\n\n"
printf "Today's date is `date`, this is week `date +"%V"`.\n\n"
printf "These users are currently connected:\n"
w | cut -d " " -f 1 - | grep -v USER | sort -u
printf "\n"
printf "This is `uname -s` running on a `uname -m` processor.\n\n"
printf "This is the uptime information:\n"
uptime
printf "\n"
printf "That's all folks!\n"
</screen>
<para>Creating user friendly scripts by means of inserting messages is treated in <xref linkend="chap_08" />.</para>
<note><title>Standard location of the Bourne Again shell</title>
<para>This implies that the <command>bash</command> program is installed in <filename>/bin</filename>.</para></note>
<warning><title>If stdout is not available</title>
<para>If you execute a script from cron, supply full path names and redirect output and errors. Since the shell runs in non-interactive mode, any errors will cause the script to exit prematurely if you don't think about this.</para></warning>
<para>The following chapters will discuss the details of the above scripts.</para>
</sect2>
<sect2 id="sect_01_05_06"><title>Example init script</title>
<para>An init script<indexterm><primary>scripts</primary><secondary>init scripts</secondary></indexterm> starts system services on UNIX and Linux machines. The system log daemon, the power management daemon, the name and mail daemons are common examples. These scripts, also known as startup scripts, are stored in a specific location on your system, such as <filename>/etc/rc.d/init.d</filename> or <filename>/etc/init.d</filename>. Init, the initial process, reads its configuration files and decides which services to start or stop in each run level. A run level is a configuration of processes; each system has a single user run level, for instance, for performing administrative tasks, for which the system has to be in an unused state as much as possible, such as recovering a critical file system from a backup. Reboot and shutdown run levels are usually also configured.</para>
<para>The tasks to be executed upon starting a service or stopping it are listed in the startup scripts. It is one of the system administrator's tasks to configure <command>init</command>, so that services are started and stopped at the correct moment. When confronted with this task, you need a good understanding of the startup and shutdown procedures on your system. We therefore advise that you read the man pages for <command>init</command> and <filename>inittab</filename> before starting on your own initialization scripts.</para>
<para>Here is a very simple example, that will play a sound upon starting<indexterm><primary>scripts</primary><secondary>init script example</secondary></indexterm> and stopping your machine:</para>
<screen>
#!/bin/bash
# This script is for /etc/rc.d/init.d
# Link in rc3.d/S99audio-greeting and rc0.d/K01audio-greeting
case "$1" in
'start')
cat /usr/share/audio/at_your_service.au &gt; /dev/audio
;;
'stop')
cat /usr/share/audio/oh_no_not_again.au &gt; /dev/audio
;;
esac
exit 0
</screen>
<para>The <command>case</command> statement often used in this kind of script is described in <xref linkend="sect_07_02_05" />.</para>
</sect2>
</sect1>
<sect1 id="sect_01_06"><title>Summary</title>
<para>Bash is the GNU shell, compatible with the Bourne shell and incorporating many useful features from other shells. When the shell is started, it reads its configuration files. The most important are:</para>
<itemizedlist>
<listitem><para><filename>/etc/profile</filename></para></listitem>
<listitem><para><filename>~/.bash_profile</filename></para></listitem>
<listitem><para><filename>~/.bashrc</filename></para></listitem>
</itemizedlist>
<para>Bash behaves different when in interactive mode and also has a POSIX compliant and a restricted mode.</para>
<para>Shell commands can be split up in three groups: the shell functions, shell built-ins and existing commands in a directory on your system. Bash supports additional built-ins not found in the plain Bourne shell.</para>
<para>Shell scripts consist of these commands arranged as shell syntax dictates. Scripts are read and executed line per line and should have a logical structure.</para>
</sect1>
<sect1 id="sect_01_07"><title>Exercises</title>
<para>These are some exercises to warm you up for the next chapter:</para>
<orderedlist>
<listitem><para>Where is the <command>bash</command> program located on your system?</para></listitem>
<listitem><para>Use the <option>--version</option> option to find out which version you are running.</para></listitem>
<listitem><para>Which shell configuration files are read when you login to your system using the graphical user interface and then opening a terminal window?</para></listitem>
<listitem><para>Are the following shells interactive shells? Are they login shells?</para>
<itemizedlist>
<listitem><para>A shell opened by clicking on the background of your graphical desktop, selecting <quote>Terminal</quote> or such from a menu.</para></listitem>
<listitem><para>A shell that you get after issuing the command <command>ssh <parameter>localhost</parameter></command>.</para></listitem>
<listitem><para>A shell that you get when logging in to the console in text mode.</para></listitem>
<listitem><para>A shell obtained by the command <command>xterm &amp;</command>.</para></listitem>
<listitem><para>A shell opened by the <command>mysystem.sh</command> script.</para></listitem>
<listitem><para>A shell that you get on a remote host, for which you didn't have to give the login and/or password because you use SSH and maybe SSH keys.</para></listitem>
</itemizedlist>
</listitem>
<listitem><para>Can you explain why <command>bash</command> does not exit when you type <keycap>Ctrl</keycap>+<keycap>C</keycap> on the command line?</para></listitem>
<listitem><para>Display directory stack content.</para></listitem>
<listitem><para>If it is not yet the case, set your prompt so that it displays your location in the file system hierarchy, for instance add this line to <filename>~/.bashrc</filename>:</para>
<cmdsynopsis><command>export <varname>PS1</varname>="<parameter>\u@\h \w&gt; </parameter>"</command></cmdsynopsis>
</listitem>
<listitem><para>Display hashed commands for your current shell session.</para></listitem>
<listitem><para>How many processes are currently running on your system? Use <command>ps</command> and <command>wc</command>, the first line of output of <command>ps</command> is not a process!</para></listitem>
<listitem><para>How to display the system hostname? Only the name, nothing more!</para></listitem>
</orderedlist>
</sect1>
</chapter>