615 lines
10 KiB
HTML
615 lines
10 KiB
HTML
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
|
|
<HTML
|
|
><HEAD
|
|
><TITLE
|
|
>Optimizations</TITLE
|
|
><META
|
|
NAME="GENERATOR"
|
|
CONTENT="Modular DocBook HTML Stylesheet Version 1.7"><LINK
|
|
REL="HOME"
|
|
TITLE="Advanced Bash-Scripting Guide"
|
|
HREF="index.html"><LINK
|
|
REL="UP"
|
|
TITLE="Miscellany"
|
|
HREF="miscellany.html"><LINK
|
|
REL="PREVIOUS"
|
|
TITLE="Colorizing Scripts"
|
|
HREF="colorizing.html"><LINK
|
|
REL="NEXT"
|
|
TITLE="Assorted Tips"
|
|
HREF="assortedtips.html"></HEAD
|
|
><BODY
|
|
CLASS="SECT1"
|
|
BGCOLOR="#FFFFFF"
|
|
TEXT="#000000"
|
|
LINK="#0000FF"
|
|
VLINK="#840084"
|
|
ALINK="#0000FF"
|
|
><DIV
|
|
CLASS="NAVHEADER"
|
|
><TABLE
|
|
SUMMARY="Header navigation table"
|
|
WIDTH="100%"
|
|
BORDER="0"
|
|
CELLPADDING="0"
|
|
CELLSPACING="0"
|
|
><TR
|
|
><TH
|
|
COLSPAN="3"
|
|
ALIGN="center"
|
|
>Advanced Bash-Scripting Guide: </TH
|
|
></TR
|
|
><TR
|
|
><TD
|
|
WIDTH="10%"
|
|
ALIGN="left"
|
|
VALIGN="bottom"
|
|
><A
|
|
HREF="colorizing.html"
|
|
ACCESSKEY="P"
|
|
>Prev</A
|
|
></TD
|
|
><TD
|
|
WIDTH="80%"
|
|
ALIGN="center"
|
|
VALIGN="bottom"
|
|
>Chapter 36. Miscellany</TD
|
|
><TD
|
|
WIDTH="10%"
|
|
ALIGN="right"
|
|
VALIGN="bottom"
|
|
><A
|
|
HREF="assortedtips.html"
|
|
ACCESSKEY="N"
|
|
>Next</A
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><HR
|
|
ALIGN="LEFT"
|
|
WIDTH="100%"></DIV
|
|
><DIV
|
|
CLASS="SECT1"
|
|
><H1
|
|
CLASS="SECT1"
|
|
><A
|
|
NAME="OPTIMIZATIONS"
|
|
></A
|
|
>36.6. Optimizations</H1
|
|
><P
|
|
>Most shell scripts are quick 'n dirty solutions to non-complex
|
|
problems. As such, optimizing them for speed is not much of an
|
|
issue. Consider the case, though, where a script carries out
|
|
an important task, does it well, but runs too slowly. Rewriting
|
|
it in a compiled language may not be a palatable option. The
|
|
simplest fix would be to rewrite the parts of the script
|
|
that slow it down. Is it possible to apply principles of code
|
|
optimization even to a lowly shell script?</P
|
|
><P
|
|
>Check the loops in the script. Time consumed by repetitive
|
|
operations adds up quickly. If at all possible, remove
|
|
time-consuming operations from within loops.</P
|
|
><P
|
|
>Use <A
|
|
HREF="internal.html#BUILTINREF"
|
|
>builtin</A
|
|
> commands in
|
|
preference to system commands. Builtins execute faster and
|
|
usually do not launch a subshell when invoked.</P
|
|
><P
|
|
><A
|
|
NAME="CATABUSE"
|
|
></A
|
|
></P
|
|
><P
|
|
>Avoid unnecessary commands, particularly in a <A
|
|
HREF="special-chars.html#PIPEREF"
|
|
>pipe</A
|
|
>.
|
|
<TABLE
|
|
BORDER="0"
|
|
BGCOLOR="#E0E0E0"
|
|
WIDTH="100%"
|
|
><TR
|
|
><TD
|
|
><FONT
|
|
COLOR="#000000"
|
|
><PRE
|
|
CLASS="PROGRAMLISTING"
|
|
>cat "$file" | grep "$word"
|
|
|
|
grep "$word" "$file"
|
|
|
|
# The above command-lines have an identical effect,
|
|
#+ but the second runs faster since it launches one fewer subprocess.</PRE
|
|
></FONT
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
>
|
|
The <A
|
|
HREF="basic.html#CATREF"
|
|
>cat</A
|
|
> command seems especially
|
|
prone to overuse in scripts.</P
|
|
><P
|
|
><A
|
|
NAME="LCALL"
|
|
></A
|
|
></P
|
|
><TABLE
|
|
CLASS="SIDEBAR"
|
|
BORDER="1"
|
|
CELLPADDING="5"
|
|
><TR
|
|
><TD
|
|
><DIV
|
|
CLASS="SIDEBAR"
|
|
><A
|
|
NAME="AEN20414"
|
|
></A
|
|
><P
|
|
></P
|
|
><P
|
|
>Disabling certain Bash options can speed up scripts.</P
|
|
><P
|
|
>As Erik Brandsberg points out:</P
|
|
><P
|
|
>If you don't need <A
|
|
HREF="bashver4.html#UNICODEREF"
|
|
>Unicode</A
|
|
> support, you can
|
|
get potentially a 2x or more improvement in speed by
|
|
simply setting the <TT
|
|
CLASS="USERINPUT"
|
|
><B
|
|
>LC_ALL</B
|
|
></TT
|
|
> variable.
|
|
|
|
<TABLE
|
|
BORDER="0"
|
|
BGCOLOR="#E0E0E0"
|
|
WIDTH="100%"
|
|
><TR
|
|
><TD
|
|
><FONT
|
|
COLOR="#000000"
|
|
><PRE
|
|
CLASS="PROGRAMLISTING"
|
|
> export LC_ALL=C
|
|
|
|
[specifies the locale as ANSI C,
|
|
thereby disabling Unicode support]
|
|
|
|
[In an example script ...]
|
|
|
|
Without [Unicode support]:
|
|
erik@erik-desktop:~/capture$ time ./cap-ngrep.sh
|
|
live2.pcap > out.txt
|
|
|
|
real 0m20.483s
|
|
user 1m34.470s
|
|
sys 0m12.869s
|
|
|
|
With [Unicode support]:
|
|
erik@erik-desktop:~/capture$ time ./cap-ngrep.sh
|
|
live2.pcap > out.txt
|
|
|
|
real 0m50.232s
|
|
user 3m51.118s
|
|
sys 0m11.221s
|
|
|
|
A large part of the overhead that is optimized is, I believe,
|
|
regex match using [[ string =~ REGEX ]],
|
|
but it may help with other portions of the code as well.
|
|
I hadn't [seen it] mentioned that this optimization helped
|
|
with Bash, but I had seen it helped with "grep,"
|
|
so why not try?</PRE
|
|
></FONT
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
></P
|
|
><P
|
|
></P
|
|
></DIV
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><P
|
|
><A
|
|
NAME="OPTIMES"
|
|
></A
|
|
></P
|
|
><DIV
|
|
CLASS="NOTE"
|
|
><P
|
|
></P
|
|
><TABLE
|
|
CLASS="NOTE"
|
|
WIDTH="100%"
|
|
BORDER="0"
|
|
><TR
|
|
><TD
|
|
WIDTH="25"
|
|
ALIGN="CENTER"
|
|
VALIGN="TOP"
|
|
><IMG
|
|
SRC="../images/note.gif"
|
|
HSPACE="5"
|
|
ALT="Note"></TD
|
|
><TD
|
|
ALIGN="LEFT"
|
|
VALIGN="TOP"
|
|
><P
|
|
>Certain operators, notably <A
|
|
HREF="moreadv.html#EXPRREF"
|
|
>expr</A
|
|
>, are very inefficient
|
|
and might be replaced by <A
|
|
HREF="dblparens.html"
|
|
>double
|
|
parentheses</A
|
|
> arithmetic expansion.
|
|
See <A
|
|
HREF="contributed-scripts.html#TESTEXECTIME"
|
|
>Example A-59</A
|
|
>.</P
|
|
><P
|
|
><TABLE
|
|
BORDER="0"
|
|
BGCOLOR="#E0E0E0"
|
|
WIDTH="100%"
|
|
><TR
|
|
><TD
|
|
><FONT
|
|
COLOR="#000000"
|
|
><PRE
|
|
CLASS="PROGRAMLISTING"
|
|
>Math tests
|
|
|
|
math via $(( ))
|
|
real 0m0.294s
|
|
user 0m0.288s
|
|
sys 0m0.008s
|
|
|
|
math via expr:
|
|
real 1m17.879s # Much slower!
|
|
user 0m3.600s
|
|
sys 0m8.765s
|
|
|
|
math via let:
|
|
real 0m0.364s
|
|
user 0m0.372s
|
|
sys 0m0.000s</PRE
|
|
></FONT
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
></P
|
|
><P
|
|
><A
|
|
HREF="tests.html#IFTHEN"
|
|
>Condition testing</A
|
|
>
|
|
constructs in scripts deserve close scrutiny. Substitute
|
|
<A
|
|
HREF="testbranch.html#CASEESAC1"
|
|
>case</A
|
|
> for <A
|
|
HREF="tests.html#IFTHEN"
|
|
>if-then</A
|
|
> constructs and combine tests
|
|
when possible, to minimize script execution time. Again,
|
|
refer to <A
|
|
HREF="contributed-scripts.html#TESTEXECTIME"
|
|
>Example A-59</A
|
|
>.</P
|
|
><P
|
|
><TABLE
|
|
BORDER="0"
|
|
BGCOLOR="#E0E0E0"
|
|
WIDTH="100%"
|
|
><TR
|
|
><TD
|
|
><FONT
|
|
COLOR="#000000"
|
|
><PRE
|
|
CLASS="PROGRAMLISTING"
|
|
>Test using "case" construct:
|
|
real 0m0.329s
|
|
user 0m0.320s
|
|
sys 0m0.000s
|
|
|
|
|
|
Test with if [], no quotes:
|
|
real 0m0.438s
|
|
user 0m0.432s
|
|
sys 0m0.008s
|
|
|
|
|
|
Test with if [], quotes:
|
|
real 0m0.476s
|
|
user 0m0.452s
|
|
sys 0m0.024s
|
|
|
|
|
|
Test with if [], using -eq:
|
|
real 0m0.457s
|
|
user 0m0.456s
|
|
sys 0m0.000s</PRE
|
|
></FONT
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
></P
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
></DIV
|
|
><P
|
|
><A
|
|
NAME="ASSOCARRTST"
|
|
></A
|
|
></P
|
|
><DIV
|
|
CLASS="NOTE"
|
|
><P
|
|
></P
|
|
><TABLE
|
|
CLASS="NOTE"
|
|
WIDTH="100%"
|
|
BORDER="0"
|
|
><TR
|
|
><TD
|
|
WIDTH="25"
|
|
ALIGN="CENTER"
|
|
VALIGN="TOP"
|
|
><IMG
|
|
SRC="../images/note.gif"
|
|
HSPACE="5"
|
|
ALT="Note"></TD
|
|
><TD
|
|
ALIGN="LEFT"
|
|
VALIGN="TOP"
|
|
><P
|
|
>Erik Brandsberg recommends using <A
|
|
HREF="bashver4.html#ASSOCARR"
|
|
>associative arrays</A
|
|
> in preference to
|
|
conventional numeric-indexed arrays in most cases. When
|
|
overwriting values in a numeric array, there is a significant
|
|
performance penalty vs. associative arrays. Running a test
|
|
script confirms this. See <A
|
|
HREF="contributed-scripts.html#ASSOCARRTEST"
|
|
>Example A-60</A
|
|
>.</P
|
|
><P
|
|
><TABLE
|
|
BORDER="0"
|
|
BGCOLOR="#E0E0E0"
|
|
WIDTH="100%"
|
|
><TR
|
|
><TD
|
|
><FONT
|
|
COLOR="#000000"
|
|
><PRE
|
|
CLASS="PROGRAMLISTING"
|
|
>Assignment tests
|
|
|
|
Assigning a simple variable
|
|
real 0m0.418s
|
|
user 0m0.416s
|
|
sys 0m0.004s
|
|
|
|
Assigning a numeric index array entry
|
|
real 0m0.582s
|
|
user 0m0.564s
|
|
sys 0m0.016s
|
|
|
|
Overwriting a numeric index array entry
|
|
real 0m21.931s
|
|
user 0m21.913s
|
|
sys 0m0.016s
|
|
|
|
Linear reading of numeric index array
|
|
real 0m0.422s
|
|
user 0m0.416s
|
|
sys 0m0.004s
|
|
|
|
Assigning an associative array entry
|
|
real 0m1.800s
|
|
user 0m1.796s
|
|
sys 0m0.004s
|
|
|
|
Overwriting an associative array entry
|
|
real 0m1.798s
|
|
user 0m1.784s
|
|
sys 0m0.012s
|
|
|
|
Linear reading an associative array entry
|
|
real 0m0.420s
|
|
user 0m0.420s
|
|
sys 0m0.000s
|
|
|
|
Assigning a random number to a simple variable
|
|
real 0m0.402s
|
|
user 0m0.388s
|
|
sys 0m0.016s
|
|
|
|
Assigning a sparse numeric index array entry randomly into 64k cells
|
|
real 0m12.678s
|
|
user 0m12.649s
|
|
sys 0m0.028s
|
|
|
|
Reading sparse numeric index array entry
|
|
real 0m0.087s
|
|
user 0m0.084s
|
|
sys 0m0.000s
|
|
|
|
Assigning a sparse associative array entry randomly into 64k cells
|
|
real 0m0.698s
|
|
user 0m0.696s
|
|
sys 0m0.004s
|
|
|
|
Reading sparse associative index array entry
|
|
real 0m0.083s
|
|
user 0m0.084s
|
|
sys 0m0.000s</PRE
|
|
></FONT
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
></P
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
></DIV
|
|
><P
|
|
>Use the <A
|
|
HREF="timedate.html#TIMREF"
|
|
>time</A
|
|
> and <A
|
|
HREF="x9644.html#TIMESREF"
|
|
>times</A
|
|
> tools to profile
|
|
computation-intensive commands. Consider rewriting time-critical
|
|
code sections in C, or even in assembler.</P
|
|
><P
|
|
>Try to minimize file I/O. Bash is not particularly
|
|
efficient at handling files, so consider using
|
|
more appropriate tools for this within the script,
|
|
such as <A
|
|
HREF="awk.html#AWKREF"
|
|
>awk</A
|
|
> or <A
|
|
HREF="wrapper.html#PERLREF"
|
|
>Perl</A
|
|
>.</P
|
|
><P
|
|
>Write your scripts in a modular and coherent form,
|
|
<A
|
|
NAME="AEN20452"
|
|
HREF="#FTN.AEN20452"
|
|
><SPAN
|
|
CLASS="footnote"
|
|
>[1]</SPAN
|
|
></A
|
|
>
|
|
so they can be reorganized and tightened up as necessary. Some
|
|
of the optimization techniques applicable to high-level
|
|
languages may work for scripts, but others, such as
|
|
<I
|
|
CLASS="FIRSTTERM"
|
|
>loop unrolling</I
|
|
>, are mostly
|
|
irrelevant. Above all, use common sense.</P
|
|
><P
|
|
>For an excellent demonstration of how optimization can
|
|
dramatically reduce the execution time of a script, see <A
|
|
HREF="mathc.html#MONTHLYPMT"
|
|
>Example 16-47</A
|
|
>.</P
|
|
></DIV
|
|
><H3
|
|
CLASS="FOOTNOTES"
|
|
>Notes</H3
|
|
><TABLE
|
|
BORDER="0"
|
|
CLASS="FOOTNOTES"
|
|
WIDTH="100%"
|
|
><TR
|
|
><TD
|
|
ALIGN="LEFT"
|
|
VALIGN="TOP"
|
|
WIDTH="5%"
|
|
><A
|
|
NAME="FTN.AEN20452"
|
|
HREF="optimizations.html#AEN20452"
|
|
><SPAN
|
|
CLASS="footnote"
|
|
>[1]</SPAN
|
|
></A
|
|
></TD
|
|
><TD
|
|
ALIGN="LEFT"
|
|
VALIGN="TOP"
|
|
WIDTH="95%"
|
|
><P
|
|
>This usually means liberal use of
|
|
<A
|
|
HREF="functions.html#FUNCTIONREF"
|
|
>functions</A
|
|
>.</P
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><DIV
|
|
CLASS="NAVFOOTER"
|
|
><HR
|
|
ALIGN="LEFT"
|
|
WIDTH="100%"><TABLE
|
|
SUMMARY="Footer navigation table"
|
|
WIDTH="100%"
|
|
BORDER="0"
|
|
CELLPADDING="0"
|
|
CELLSPACING="0"
|
|
><TR
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="left"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="colorizing.html"
|
|
ACCESSKEY="P"
|
|
>Prev</A
|
|
></TD
|
|
><TD
|
|
WIDTH="34%"
|
|
ALIGN="center"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="index.html"
|
|
ACCESSKEY="H"
|
|
>Home</A
|
|
></TD
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="right"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="assortedtips.html"
|
|
ACCESSKEY="N"
|
|
>Next</A
|
|
></TD
|
|
></TR
|
|
><TR
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="left"
|
|
VALIGN="top"
|
|
><SPAN
|
|
CLASS="QUOTE"
|
|
>"Colorizing"</SPAN
|
|
> Scripts</TD
|
|
><TD
|
|
WIDTH="34%"
|
|
ALIGN="center"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="miscellany.html"
|
|
ACCESSKEY="U"
|
|
>Up</A
|
|
></TD
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="right"
|
|
VALIGN="top"
|
|
>Assorted Tips</TD
|
|
></TR
|
|
></TABLE
|
|
></DIV
|
|
></BODY
|
|
></HTML
|
|
> |