From ae3ef220808545ccbad5b71a978e642f36999a9b Mon Sep 17 00:00:00 2001 From: gferg <> Date: Mon, 12 Jul 2004 13:08:16 +0000 Subject: [PATCH] updated --- LDP/guide/docbook/abs-guide/Change.log | 74 + LDP/guide/docbook/abs-guide/abs-guide.sgml | 374 ++- LDP/guide/docbook/abs-guide/ex27.sh | 8 +- LDP/guide/docbook/abs-guide/ex29.sh | 14 +- LDP/guide/docbook/abs-guide/ex73.sh | 4 +- LDP/guide/docbook/abs-guide/ex79.sh | 18 +- LDP/howto/docbook/HOWTO-INDEX/howtoChap.sgml | 2 +- LDP/howto/docbook/HOWTO-INDEX/miscSect.sgml | 2 +- .../Scientific-Computing-with-GNU-Linux.xml | 2883 +++++++++-------- 9 files changed, 1867 insertions(+), 1512 deletions(-) diff --git a/LDP/guide/docbook/abs-guide/Change.log b/LDP/guide/docbook/abs-guide/Change.log index 4202d4e0..7342c988 100644 --- a/LDP/guide/docbook/abs-guide/Change.log +++ b/LDP/guide/docbook/abs-guide/Change.log @@ -6,6 +6,80 @@ http://personal.riverusers.com/~thegrendel/Change.log ------------------------------------------------------------------------ +Version 2.8 +Elderberry release, 07/11/04 + +1) In the introduction: + Added link to pdf version of the book. + +2) In "Arithmetic Expansion" chapter: + Fixed typo -- missing parenthesis -- in inline example. + (Thank you, Thorsten Bonow.) + +3) In "Special Characters" chapter: + Added material at "Ctl-D" entry. + Added material at "Ctl-K" entry. + Added material at "Ctl-U" entry. + (Thank you, Jeremy Impson.) + Added "Ctl-V" entry. + (Thank you, Jeremy Impson.) + Added "Ctl-W" entry. + (Thank you, Jeremy Impson.) + At "-" (redirection from/to stdin or stdout) entry: + Fixed comment in Alan Cox's example. + (Thank you, Ken Fuchs.) + +4) In "Gotchas" chapter: + Added another alternative to Anthony Richardson's piping-loop example. + (Thank you, Wilbert Berendsen.) + +5) In "Internal Commands and Builtins" chapter: + At "echo" entry, showed how to embed a linefeed within echoed text. + (Thanks, Steve Parker.) + At "read" entry, added note that "read -n" command sequence will not + detect the ENTER (newline) key. + (Thank you, Tony Morgan.) + At "source" entry, rewrote introductory paragraph for additional + clarity. + Added "bind" entry, with a brief footnote about the readline library. + Slight modification of discussion at "help" entry. + +6) In "File and Archiving Commands" section of "External Commands" Chapter: + At "sum, cksum, md5sum" entry, added usage examples and expanded note. + +7) In "List Constructs" chapter: + Added example snippets from Miquel van Smoorenburg's + /etc/rc.d/init.d/single script. + +8) In "/dev" section of "/dev and /proc" chapter: + Fixed up "Downloading a URL" inline example. + (Thanks, Mihai Maties.) + Added "noatime" to USB flash drive installation example. + +9) Cleaned up and made consistent notes about #!/bin/sh disabling extended + Bash functionality. + +10) In "Exit Codes With Special Meanings" appendix: + Fixed footnote about out-of-range exit values. + +11) In "Important System Directories" appendix: + Added /var and /boot entries. + Additional short notes. + Other Minor changes. + +12) In the "Sed and Awk Micro-primer" appendix: + Corrected explanation of backslash in sed. + (Thank you, George Cristian Birzan.) + +13) In "Bibliography" section: + Added entry for "The Linux USB subsystem." + +14) Various minor changes to example scripts. + + + + + Version 2.7 Mulberry release, 04/18/04 diff --git a/LDP/guide/docbook/abs-guide/abs-guide.sgml b/LDP/guide/docbook/abs-guide/abs-guide.sgml index 42c8c2f1..6d0ff5c9 100644 --- a/LDP/guide/docbook/abs-guide/abs-guide.sgml +++ b/LDP/guide/docbook/abs-guide/abs-guide.sgml @@ -322,8 +322,8 @@ Uncomment line below to generate index. - 2.7 - 18 April 2004 + 2.8 + 11 July 2004 @@ -368,12 +368,14 @@ Uncomment line below to generate index. This book is suitable for classroom use as a general introduction to programming concepts. - The latest update of this document, as an archived, + The latest update of this document, as an archived, bzip2-ed tarball - including both the SGML source and - rendered HTML, may be downloaded from - the author's home site. See the pdf + version is also available. See the change log for a revision history. @@ -687,7 +689,7 @@ exit $WHATEVER # Doesn't matter. The script will not exit here.#!/bin/sh, the default Bourne shell in most commercial variants of Unix, makes the script portable to non-Linux machines, - though you may have to sacrifice a few Bash-specific features. + though you sacrifice Bash-specific features. The script will, however, conform to the POSIX @@ -781,7 +783,7 @@ fi scriptname? If the directory you are in ($PWD) is where scriptname is located, why doesn't this - work? This fails because, for security reasons, the current + work? This fails because, for security reasons, the current directory, . is not included in a user's $PATH. It is therefore necessary to explicitly invoke the script in the current directory with @@ -2302,7 +2304,7 @@ fi - - + - - @@ -2342,10 +2344,10 @@ fi # More elegant than, but equivalent to: -# cd source-directory -# tar cf - . | (cd ../target-directory; tar xzf -) +# cd source/directory +# tar cf - . | (cd ../dest/directory; tar xpvf -) # -# cp -a /source/directory /dest also has same effect. +# cp -a /source/directory /dest/directory also has same effect. bunzip2 linux-2.4.3.tar.bz2 | tar xvf - @@ -2607,7 +2609,10 @@ echo $a # 28 terminal or text display. A control character is a CONTROL + key combination. - + + Control characters are not normally useful inside a + script. + @@ -2622,12 +2627,18 @@ echo $a # 28 + Ctl-D Log out from a shell (similar to exit). EOF (end of file). This also terminates input from stdin. + When typing text on the console or in an xterm window, + Ctl-D erases the character under the + cursor. When there are no characters present, + Ctl-D logs out of the session, as + expected. @@ -2666,6 +2677,9 @@ echo; echo Ctl-K Vertical tab. + When typing text on the console or in an xterm window, + Ctl-K erases from the character + under the cursor to end of line. @@ -2734,7 +2748,31 @@ exit 0 Ctl-U - Erase a line of input. + Erase a line of input, from the cursor backward to + beginning of line. In some settings, + Ctl-U erases the entire + line of input, regardless of cursor + position. + + + + Ctl-V + When inputting text, Ctl-V + permits inserting control characters. For example, the + following two are equivalent: + echo -e '\x0a' +echo <Ctl-V><Ctl-J> + Ctl-V is primarily useful from + within a text editor. + + + + Ctl-W + When typing text on the console or in an xterm window, + Ctl-W erases from the character + under the cursor backwards to the first instance of + whitespace. In some settings, Ctl-W + erases backwards to first non-alphanumeric character. @@ -8897,6 +8935,54 @@ fi + + So, how can we embed a linefeed within an + echoed character string? + + # Embedding a linefeed? +echo "Why doesn't this string \n split on two lines?" +# Doesn't split. + +# Let's try something else. + +echo + +echo $"A line of text containing +a linefeed." +# Prints as two distinct lines (embedded linefeed). +# But, is the "$" variable prefix really necessary? + +echo + +echo "This string splits +on two lines." +# No, the "$" is not needed. + +echo +echo "---------------" +echo + +echo -n $"Another line of text containing +a linefeed." +# Prints as two distinct lines (embedded linefeed). +# Even the -n option fails to suppress the linefeed here. + +echo +echo +echo "---------------" +echo +echo + +# However, the following doesn't work as expected. +# Why not? +string1=$"Yet another line of text containing +a linefeed (maybe)." + +echo $string1 + +# Thanks, Steve Parker, for pointing this out. + + This command is a shell builtin, and not the same as /bin/echo, although its behavior is @@ -9036,6 +9122,10 @@ echo; echo "Keypress was "\"$keypress\""." &arrowdetect; + The option to read + will not detect the ENTER (newline) + key. + The option to read permits timed input (see ). @@ -9688,11 +9778,15 @@ shift $(($OPTIND - 1)) This command, when invoked from the command line, executes a script. Within a script, a - source file-name loads the file - file-name. This is the shell scripting - equivalent of a C/C++ #include - directive. It is useful in situations when multiple scripts - use a common data file or function library. + source file-name loads the + file file-name. Sourcing a file + (dot-command) imports + code into the script, appending to the script (same + effect as the #include directive + in a C program). The net result is the same as if the + sourced lines of code were in the body of + the script. This is useful in situations when multiple + scripts use a common data file or function library. <quote>Including</quote> a data file @@ -9940,6 +10034,28 @@ done + + + bind + + bind + + + bind + key bindings + + + + The bind builtin displays or modifies + readline + The readline library + is what Bash uses for reading input in an interactive + shell. + key bindings. + + + + help @@ -9951,8 +10067,7 @@ done - help COMMAND looks up - a short usage summary of the shell builtin COMMAND. This is + Gets a short usage summary of a shell builtin. This is the counterpart to whatis, but for builtins. @@ -10423,7 +10538,7 @@ wait to underestimate the power of this humble command. For example, using the , recursive option, ls provides a tree-like listing of - a directory structure. Other interesting options are + a directory structure. Other useful options are , sort listing by file size, , sort by file modification time, and , show file inodes (see bash$ cksum /boot/vmlinuz 1670054224 804083 /boot/vmlinuz +bash$ echo -n "Top Secret" | cksum +3391003827 10 + + bash$ md5sum /boot/vmlinuz 0f43eccea8f09e0a0b2b5cf1dcf333ba /boot/vmlinuz + +bash$ echo -n "Top Secret" | md5sum +8babc97a6f62a4649716f4df8d61728f - - Note that cksum also shows the size, - in bytes, of the target file. + + The cksum command shows the size, + in bytes, of its target, whether file or + stdout. + The md5sum command displays a + dash when it receives its input from + stdout. + Checking file integrity @@ -18934,7 +19062,7 @@ File_contents2=$(<$file2) # Bash permits this also. z=$(($z+3)) -z=$((z+3) # Also correct. +z=$((z+3)) # Also correct. # Within double parentheses, #+ parameter dereferencing #+ is optional. @@ -21460,6 +21588,8 @@ drwxr-xr-x 40 bozo bozo 2048 Feb 6 14:04 .. false is the last one to execute). + + Using an <quote>and list</quote> to test for command-line arguments &ex64; @@ -21509,6 +21639,38 @@ drwxr-xr-x 40 bozo bozo 2048 Feb 6 14:04 .. + + # ==> The following snippets from the /etc/rc.d/init.d/single script by Miquel van Smoorenburg +#+==> illustrate use of "and" and "or" lists. +# ==> "Arrowed" comments added by document author. + +[ -x /usr/bin/clear ] && /usr/bin/clear + # ==> If /usr/bin/clear exists, then invoke it. + # ==> Checking for the existence of a command before calling it + #+==> avoids error messages and other awkward consequences. + + # ==> . . . + +# If they want to run something in single user mode, might as well run it... +for i in /etc/rc1.d/S[0-9][0-9]* ; do + # Check if the script is there. + [ -x "$i" ] || continue + # ==> If corresponding file in $PWD *not* found, + #+==> then "continue" by jumping to the top of the loop. + + # Reject backup files and files generated by rpm. + case "$1" in + *.rpmsave|*.rpmorig|*.rpmnew|*~|*.orig) + continue;; + esac + [ "$i" = "/etc/rc1.d/S00single" ] && continue + # ==> Set script name, but don't execute it yet. + $i start +done + + # ==> . . . + + The exit status of an and list or an or list is the exit status of the last @@ -21988,7 +22150,7 @@ exit 0 mount /mnt/flashdrive - /dev/sda1 /mnt/flashdrive auto noauto,user 0 0 + /dev/sda1 /mnt/flashdrive auto noauto,user,noatime 0 0 (See also .) @@ -22013,14 +22175,16 @@ exit 0 53082 04-03-18 04:26:54 68 0 0 502.3 UTC(NIST) * + [Mark contributed the above example.] + Downloading a URL: bash$ exec 5<>/dev/tcp/www.slashdot.org/80 -bash$ echo "GET / HTTP/1.0 -> " >&5" +bash$ echo -e "GET / HTTP/1.0\n" >&5 bash$ cat <&5 - [Mark contributed the last two examples.] + [Thanks, Mark and Mihai Maties.] + @@ -22948,7 +23112,7 @@ exit 0 A shell script headed by #!/bin/sh - may not run in full Bash-compatibility mode. Some Bash-specific + will not run in full Bash-compatibility mode. Some Bash-specific functions might be disabled. Scripts that need complete access to all the Bash-specific extensions should start with #!/bin/bash. @@ -23001,7 +23165,8 @@ exit 0 # Loop piping troubles. -# This example by Anthony Richardson. +# This example by Anthony Richardson, +#+ with addendum by Wilbert Berendsen. foundone=false @@ -23034,7 +23199,28 @@ done if [ $foundone = false ] then echo "No files need archiving." -fi +fi + +# ==================And here is another alternative================== + +# Places the part of the script that reads the variables +#+ within a code block, so they share the same subshell. +# Thank you, W.B. + +find $HOME -type f -atime +30 -size 100k | { + foundone=false + while read f + do + echo "$f is over 100KB and has not been accessed in over 30 days" + echo "Consider moving the file to archives." + foundone=true + done + + if ! $foundone + then + echo "No files need archiving." + fi +} @@ -24299,9 +24485,10 @@ fi #!/bin/sh header in the script, rather than #!/bin/bash - Of course, /bin/sh is a /bin/sh is a link to /bin/bash - in Linux and certain other flavors of UNIX. + in Linux and certain other flavors of UNIX, and a script invoked + this way disables extended Bash funtionality. Most Bash scripts will run as-is under ksh, and vice-versa, since Chet Ramey has @@ -24659,19 +24846,20 @@ fi errors and other corrections. Special thanks! Others contributing scripts, making helpful suggestions, and - pointing out errors were Gabor Kiss, Leopold Toetsch, Peter - Tillier, Marcus Berglof, Tony Richardson, Nick Drage (script - ideas!), Rich Bartell, Jess Thrysoee, Adam Lazur, Bram Moolenaar, - Baris Cicek, Greg Keraunen, Keith Matthews, Sandro Magi, - Albert Reiner, Dim Segebart, Rory Winston, Lee Bigelow, Wayne - Pollock, jipe, Mark, Emilio Conti, - Ian. D. Allen, Arun Giridhar, Dennis Leeuw, Dan Jacobson, Aurelio - Marinho Jargas, Edward Scholtz, Jean Helou, Chris Martin, Lee - Maschmeyer, Bruno Haible, Sebastien Godard, Bjön Eriksson, - nyal, John MacDonald, Joshua Tschida, Troy Engel, - Manfred Schwarb, Amit Singh, Bill Gradwohl, David Lombard, - Jason Parker, Bruce W. Clare, William Park, Vernia Damiano, - and David Lawyer (himself an author of four HOWTOs). + pointing out errors were Gabor Kiss, Leopold Toetsch, + Peter Tillier, Marcus Berglof, Tony Richardson, Nick Drage + (script ideas!), Rich Bartell, Jess Thrysoee, Adam Lazur, Bram + Moolenaar, Baris Cicek, Greg Keraunen, Keith Matthews, Sandro + Magi, Albert Reiner, Dim Segebart, Rory Winston, Lee Bigelow, + Wayne Pollock, jipe, Mark, Emilio + Conti, Ian. D. Allen, Arun Giridhar, Dennis Leeuw, Dan Jacobson, + Aurelio Marinho Jargas, Edward Scholtz, Jean Helou, Chris Martin, + Lee Maschmeyer, Bruno Haible, Wilbert Berendsen, Sebastien Godard, + Bjön Eriksson, nyal, John MacDonald, Joshua + Tschida, Troy Engel, Manfred Schwarb, Amit Singh, Bill Gradwohl, + David Lombard, Jason Parker, Steve Parker, Bruce W. Clare, + William Park, Vernia Damiano, Mihai Maties, Jeremy Impson, Ken + Fuchs, and David Lawyer (himself an author of four HOWTOs). My gratitude to Chet Ramey and Brian Fox for writing Bash, @@ -25252,6 +25440,14 @@ fi + + + The + Linux USB subsystem (helpful in writing scripts affecting + USB peripherals). + + + There is some nice material on William Park is working on a project to - incorporate certain Awk and Python features into Bash. + incorporate certain Awk and Python features into Bash. Among + these is a gdbm interface. @@ -26460,14 +26657,17 @@ pattern=BEGIN results in The most important parts of any application are its and sound effects - The backslash represents a newline as a - substitution character. In this special case, the replacement - expression continues on the next line. + A backslash forces the sed replacement + command to continue on to the next line. This has the effect of + using the newline at the end of the first + line as the replacement string. + s/^ */\ /g + This substitution replaces line-beginning spaces with a - newline. The net result is to replace paragraph indents with - a blank line between paragraphs. + newline. The net result is to replace paragraph indents with a + blank line between paragraphs. An address range followed by one or more operations may require open and closed curly brackets, with appropriate newlines. @@ -26643,7 +26843,7 @@ awk '{print $1 $5 $6}' $filename invalid argument to exit exit 3.14159 exit takes only integer args in the - range 0 - 255 + range 0 - 255 (see footnote) @@ -26672,9 +26872,10 @@ awk '{print $1 $5 $6}' $filename According to the table, exit codes 1 - 2, 126 - 165, and 255 Out of range exit values can result in - unpredictable exit codes. For example, - exit 3809 gives an exit code of - 225. + unexpected exit codes. An exit value greater than 255 returns an + exit code modulo 256. For example, exit 3809 + gives an exit code of 225 (3809 % 256 = + 225). have special meanings, and should therefore be avoided as user-specified exit parameters. Ending a script with exit @@ -26984,8 +27185,8 @@ exit 0 /bin - Binary executables. Basic system programs and utilities - (such as bash). + Binaries (executables). Basic system programs + and utilities (such as bash). @@ -26995,7 +27196,7 @@ exit 0 Some early Unix systems had a fast, small-capacity fixed disk (containing /, the root partition), and a second drive which - was larger but slower (containing /usr and other partitions). The most frequently used programs and utilities therefore resided on the small-but-fast @@ -27012,25 +27213,25 @@ exit 0 - More system executables. + More system binaries. /usr/local/bin - Miscellaneous executables. + Miscellaneous binaries local to the particular machine. /sbin - Superuser binaries. Basic system administrative programs + System binaries. Basic system administrative programs and utilities (such as fsck). /usr/sbin - More superuser binaries. + More system administrative programs and utilities. @@ -27050,11 +27251,31 @@ exit 0 Documentation for installed packages. + + /usr/man + The systemwide manpages. + + /tmp System temporary files. + + /sys + Systemwide process directory. Contains information and + statistics about running processes. This is newly added to Linux + with the 2.6.X kernels. + + + + /var + Variable (changeable) system + files. This is a catchall scratchpad + directory for data generated while a Linux/UNIX machine + is running. + + /var/log Systemwide log files. @@ -27065,6 +27286,24 @@ exit 0 User mail spool. + + /lib + Systemwide library files. + + + + /usr/lib + More systemwide library files. + + + + /boot + System boot directory. The kernel, + module links, system map, and boot manager reside here. + Altering files in this directory may result in an + unbootable system. + + @@ -28690,6 +28929,11 @@ fairly detailed rundown on the Playfair Cipher and its solution methods.18 Apr 2004 MULBERRY release: Minor update. + + + 11 Jul 2004 + ELDERBERRY release: Minor update. + diff --git a/LDP/guide/docbook/abs-guide/ex27.sh b/LDP/guide/docbook/abs-guide/ex27.sh index 487d0bc0..4b3b727b 100644 --- a/LDP/guide/docbook/abs-guide/ex27.sh +++ b/LDP/guide/docbook/abs-guide/ex27.sh @@ -1,11 +1,15 @@ #!/bin/bash -until [ "$var1" = end ] # Tests condition here, at top of loop. +END_CONDITION=end + +until [ "$var1" = "$END_CONDITION" ] +# Tests condition here, at top of loop. do echo "Input variable #1 " - echo "(end to exit)" + echo "($END_CONDITION to exit)" read var1 echo "variable #1 = $var1" + echo done exit 0 diff --git a/LDP/guide/docbook/abs-guide/ex29.sh b/LDP/guide/docbook/abs-guide/ex29.sh index 74507fe5..770b723b 100644 --- a/LDP/guide/docbook/abs-guide/ex29.sh +++ b/LDP/guide/docbook/abs-guide/ex29.sh @@ -8,13 +8,13 @@ case "$Keypress" in [A-Z] ) echo "Uppercase letter";; [0-9] ) echo "Digit";; * ) echo "Punctuation, whitespace, or other";; -esac # Allows ranges of characters in [square brackets]. +esac # Allows ranges of characters in [square brackets]. -# Exercise: -# -------- -# As the script stands, # it accepts a single keystroke, then terminates. -# Change the script so it accepts continuous input, -# reports on each keystroke, and terminates only when "X" is hit. -# Hint: enclose everything in a "while" loop. +# Exercise: +# -------- +# As the script stands, it accepts a single keystroke, then terminates. +# Change the script so it accepts continuous input, +#+ reports on each keystroke, and terminates only when "X" is hit. +# Hint: enclose everything in a "while" loop. exit 0 diff --git a/LDP/guide/docbook/abs-guide/ex73.sh b/LDP/guide/docbook/abs-guide/ex73.sh index 7e1a6b5c..c54820de 100644 --- a/LDP/guide/docbook/abs-guide/ex73.sh +++ b/LDP/guide/docbook/abs-guide/ex73.sh @@ -1,7 +1,5 @@ #!/bin/bash - # Creating a swapfile. -# This script must be run as root. ROOT_UID=0 # Root has $UID 0. E_WRONG_USER=65 # Not root? @@ -11,6 +9,8 @@ BLOCKSIZE=1024 MINBLOCKS=40 SUCCESS=0 + +# This script must be run as root. if [ "$UID" -ne "$ROOT_UID" ] then echo; echo "You must be root to run this script."; echo diff --git a/LDP/guide/docbook/abs-guide/ex79.sh b/LDP/guide/docbook/abs-guide/ex79.sh index e96d144f..b00584ba 100644 --- a/LDP/guide/docbook/abs-guide/ex79.sh +++ b/LDP/guide/docbook/abs-guide/ex79.sh @@ -2,7 +2,7 @@ # May need to be invoked with #!/bin/bash2 on older machines. # Cards: -# deals four random hands from a deck of cards. +# Deals four random hands from a deck of cards. UNPICKED=0 PICKED=1 @@ -17,9 +17,9 @@ CARDS=52 declare -a Deck declare -a Suits declare -a Cards -# It would have been easier and more intuitive -# with a single, 3-dimensional array. -# Perhaps a future version of Bash will support multidimensional arrays. +# It would have been easier and more intuitive +#+ with a single, 3-dimensional array. +# Perhaps a future version of Bash will support multidimensional arrays. initialize_Deck () @@ -73,7 +73,7 @@ printf %-4s $Card } seed_random () # Seed random number generator. -{ +{ # What happens if you don't do this? seed=`eval date +%s` let "seed %= 32766" RANDOM=$seed @@ -94,7 +94,7 @@ do parse_card $t u=$cards_picked+1 - # Change back to 1-based indexing (temporarily). + # Change back to 1-based indexing (temporarily). Why? let "u %= $CARDS_IN_SUIT" if [ "$u" -eq 0 ] # Nested if/then condition test. then @@ -114,7 +114,7 @@ return 0 # Structured programming: -# entire program logic modularized in functions. +# Entire program logic modularized in functions. #================ seed_random @@ -122,9 +122,9 @@ initialize_Deck initialize_Suits initialize_Cards deal_cards +#================ exit 0 -#================ @@ -132,7 +132,7 @@ exit 0 # Add comments to thoroughly document this script. # Exercise 2: -# Revise the script to print out each hand sorted in suits. +# Add a routine (function) to print out each hand sorted in suits. # You may add other bells and whistles if you like. # Exercise 3: diff --git a/LDP/howto/docbook/HOWTO-INDEX/howtoChap.sgml b/LDP/howto/docbook/HOWTO-INDEX/howtoChap.sgml index 3dafd16f..1ffd36a3 100644 --- a/LDP/howto/docbook/HOWTO-INDEX/howtoChap.sgml +++ b/LDP/howto/docbook/HOWTO-INDEX/howtoChap.sgml @@ -4023,7 +4023,7 @@ scanner device on a system running Linux. Scientific-Computing-with-GNU-Linux, Scientific Computing with Free GNU/Linux Software HOWTO -Updated: Nov 2003. +Updated: Jun 2004. Aims to show how a PC running GNU/Linux can be used for scientific computing. It lists the various available free software and also links on the world wide web to tutorials diff --git a/LDP/howto/docbook/HOWTO-INDEX/miscSect.sgml b/LDP/howto/docbook/HOWTO-INDEX/miscSect.sgml index 3f4647af..3985d606 100644 --- a/LDP/howto/docbook/HOWTO-INDEX/miscSect.sgml +++ b/LDP/howto/docbook/HOWTO-INDEX/miscSect.sgml @@ -420,7 +420,7 @@ software from John-Paul Roubelat, F6FBB. Scientific-Computing-with-GNU-Linux, Scientific Computing with Free GNU/Linux Software HOWTO -Updated: Nov 2003. +Updated: Jun 2004. Aims to show how a PC running GNU/Linux can be used for scientific computing. It lists the various available free software and also links on the world wide web to tutorials diff --git a/LDP/howto/docbook/Scientific-Computing-with-GNU-Linux.xml b/LDP/howto/docbook/Scientific-Computing-with-GNU-Linux.xml index 096e91ea..710e3548 100644 --- a/LDP/howto/docbook/Scientific-Computing-with-GNU-Linux.xml +++ b/LDP/howto/docbook/Scientific-Computing-with-GNU-Linux.xml @@ -1,1425 +1,1458 @@ - - - -
- - - - Scientific Computing with Free GNU/Linux Software HOWTO - - - Manoj - Warrier - - -
m_war (at) users.sourceforge.net
- - - - Shishir - Deshpande - - -
shishir (at) ipr.res.in
- - - - V. S. - Ashoka - - -
ashok (at) rri.res.in
- - - 2003-10-03 - - - - - - 1.0 - 2003-11-18 - JP - Document Reviewed by LDP. - - - - - - - 0.0 - 2003-10-01 - M. W - first draft proposed - - - - - - - - - This document aims to show how a PC running GNU/Linux can be used - for scientific computing. It lists the various available free software - and also links on the world wide web to tutorials on getting started - with the tools. - - - - - - - Preamble - - - - Copyright and License - - - - This document, Scientific Computing with free software on GNU/Linux HOWTO, - is copyrighted (c) 2002 by Manoj Warrier. - Permission is granted to copy, distribute and/or modify this document under the terms of the - GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; - with no Invariant Sections, - with no Front-Cover Texts, and with no Back-Cover Texts. - A copy of the license is available - - here - - - - - Disclaimer - - - No liability for the contents of this document is accepted. - Use of the concepts, examples, links and information is entirely at - your own risk. - There may be errors and inaccuracies, that could damage your system, - waste your time, etc... Proceed with caution, and although - this is unlikely, the author takes no responsibility whatsoever. - - - - All copyrights are held by their respective owners, unless - specifically noted otherwise. Use of a term in this document - should not be regarded as affecting the validity of any trademark - or service mark. Naming of particular products, software or - brands should not be seen as endorsements - - - - I have not used many of the software applications to which links are provided. - There are simply too many applications that do the same thing, - that one cannot be expected to have used all of them. In a book - on Scientific Computing using GNU/Linux, one would - mention ones favorite tool to carry out a task and describe it in - detail. However this is a howto providing links to various - available free tools for scientific computing and may contain - links to some software that promises much but delivers little - and vice versa. - - - - - - Motivation - - This howto mainly consists of the links provided at - http://Scilinux.sourceforge.net which has to be disbanded due to a name - conflict. The best alternative seems to be to make it a Linux document - and host it at the LDP site. Another reason is that there seems to be - many free software applications doing the same things. We hope to provide links to - the available software thereby making it easy for the scientific community - to make a choice without spending much time. - - - - - - Credits / Contributors - - - In this document, I have the pleasure of acknowledging: - - - - - - Linus Trovalds, Richard M. Stallman and their merry men for - Linux, GNU and also for indirectly broadening various - perspectives which were not really obvious. - - - - - A host of colleagues and friends from the - Institute for Plasma Research, India - for discussions at various times. - - - - - Marcel Bose, Ivan Lamouret, K. Scott Hunziker, Livine Christin, - W. Herbert, Simon Pinches and many others for suggesting various - links mentioned in this document. - - - - - Vasudha my wife for letting me do what I wish and egging me on - with comments like "let us hope that you will finish at least - this project" - - - - - Shishir and Ashoka are co-authors of this document because such a - collection of links was Shishir's idea and Ashoka is always - contributing by providing links, suggestions and a second point - of view. They will be helping me maintain this HOWTO too. - - - - - - Feedback - - Feedback is most certainly welcome for this document. Send - your additions, comments and criticisms to the following - email address : m_war at users.sourceforge.net. - - - - - - Translations - - No translations yet. - - - - - - - - - Introduction - - GNU/Linux is probably the platform of choice for scientific - computing. There exists a wide variety of high level languages, - debugging tools and other code development tools for - programming, numerical subroutines for solving various types of - equations, plotting and visualization packages, word processing - software which can display equations and figures and in fact parallel - programming software to construct a supercomputer with off the shelf - PC parts and some hardware. This document aims to provide a list - of free software for carrying out the above - tasks and links to tutorials and other documents on how to setup and - use these software applications. - - - - This document does not aim to provide links to subject specific free - software available for GNU/Linux systems. It aims to show how - GNU/Linux can be used best to handle scientific computing tasks. - It is hoped that people or institutions with interest in a - specific subject list, compile a list of the free software available - for that subject ... for example see Linux for Astronomy, - Linux for Biotechnology and Linux for - Chemistry at - The Random Factory - . Another site with a lot of links (to commercial and - free) scientific software is - Scientific Applications - on Linux. The - GNU Software Directory also has links to many of the links - provided in this howto plus many more topic specific software. - - - - The software links provided are classified into - - - Code development tools - - - Mathematics packages - - - Numerical subroutines and libraries - - - Graphics and visualization - - - GNU/Linux Systems - - - Publishing tools - - - Databases - - - Linux in the Laboratory - - - - - - Just installing GNU/Linux on your PC makes it a powerful workstation. - - The various popular distributions however do not have all the tools - needed to make it the ideal scientific computing machine. This HOWTO - aims to fill in this gap by creating a list of free software useful - for scientific computing. It is assumed that people reading this - document already have a PC with Linux and the GNU utilities installed. - For those who do not have such a setup and want to install GNU-Linux - can check out GNU/Linux Systems - for links to documents on installing GNU/Linux, and also on how to get - started using GNU/Linux. Recently there has been an effort by Dirk - Eddelbuettel to create a scientific computing environment - Quantian - which probably is the first GNU-Linux distribution tailored for - Scientists. I do hope other distributions too start providing these - software applications. - - - - - Code Development Tools - - Code development consists of mainly Programming languages, - Debugging tools, Version Management tools, Compiling tools, - and Integrated Development Environments (IDEs) where all - the above are coupled as a single software application. - - - - - Programming Languages - - Links are provided to various compilers used in Scientific - Computing like FORTRAN, C, C++, Java and more recently - Python. The FORTRAN links are to compilers for FORTRAN-77 - and to the best of my knowledge there is no open source - free compiler available for FORTRAN-90 (F90) though an effort is - on to create a free, open-source Fortran 95 compiler - at g95. - - - - - - GNU Compiler Collection - : - GNU's project to produce a world class optimizing compiler. - It works on multiple architectures and diverse environments. - Currently GCC contains front ends for C, C++, Objective C, - FORTRAN, Java, and Ada, as well as libraries for these languages - (libstdc++, libgcj,..). - - - For manuals on using the various GCC compilers check out - - The GCC online documentation - - - - - - g77 - : - The GCC front end for FORTRAN 77. It is a very good FORTRAN77 - compiler. It however does not have the -r8 option which compiles - a program as double precision. This could be a good compiler - design philosophy but in many cases gives problems when porting - a code from SUN / DEC / HP workstations onto Linux systems. - The g77 manual is available at - The Gcc Online - documentation site. - - - - - - - fort77 and f2c: - fort77 is a perl program which invokes the f2c command - (a Fortran to C translator) transparently, so it can - be used just like a real Fortran compiler. Fort77 can - be used to compile Fortran, C and assembler code and - can link the code with f2c libraries. If you install - fort77, you'll also need to install the f2c package. - This does not have the "-r8" problem. You can download - fort77 and f2c from the above link. - - - - - - lush: - An object-oriented programming language, which combines the flexibility - of an interpretive language, with the efficiency of a compiled language. - It has full interfaces to numerical libraries (GSL, LAPACK, BLAS), - graphics libraries (OpenGL), which allows creation of graphics and 3D - animations and many other features that sound too good to be - true. I have not yet tried this out, but it sounds very promising. - - - - - - Scientific Python: - You may want to explore - Python for your - scientific computing needs. Python is an interpreted, - interactive, object-oriented programming language. - - It has a number of extensions for numerics, plotting, - data storage and combined with Tk lets you develop very - good GUIs for your codes. The most exciting aspect is that - it simplifies programming because it has modules for almost - anything (vectors, tensors, transformations, derivatives, - linear algebra, Fourier transforms, statistics, etc ...) are - available. You can also wrap C and Fortran libraries from - Python. Finally if you want to write a numerical scheme of - your own you may find that it is simpler in Python. - There are also interfaces to netCDF (portable binary files), - MPI and BSPlib (parallel programming). - - - You can further explore Python for Scientific computing here: - - - - - - Scientific-Python: - A collection of modules for scientific computing on Python. - All the necessary modules can be downloaded as either a tar file - or an RPM file from here. The maintainer Konrad HINSEN also has - a nice tutorial on - - Scientific Computing in Python. - - - - - SciPy An open source - library of scientific tools for Python. It includes modules for - graphics and plotting, optimization, integration, special - functions, signal and image processing, genetic algorithms, - ODE solvers, etc. - - - - - - - - - - - Debugging Tools - - In this section links are given to mainly debugging tools for GCC and - FORTRAN. I understand that python has a debugging module built in though I - have not used it. The purpose of a debugger is to allow you to see what is - going on inside a program while it executes or what the program was doing - when/if it crashed. - - - - Ftnchek: - A FORTRAN checker designed to detect errors in a Fortran - program that a compiler usually does not. Therefore it is best - to run ftnchek on your FORTRAN programs after it has compiled - without errors. Its purpose is to assist the user in finding - semantic errors. Semantic errors are legal in the Fortran - language but are wasteful or may cause incorrect operation. - An on-line - manual is available. This project is looking for volunteers - to bringing ftnchek up to the Fortran 90 standard. - - - - gdb - : - All programs written in the languages supported by GCC can be - debugged using gdb, an excellent interactive, - command line debugger. You can compile your programs using a -g option - which then compiles your code with debugging information inserted - into the executable. It can start your programs, stop your programs - on specified conditions and at specified locations, examine what - happened when your program stops. In a large code with multiple - cascading calls to various functions it can back trace the function - calls. You can also - Download - the document Debugging with GDB and - a quick reference card. - - - - xxgdb: - It is a front end to the gdb debugger. Useful for beginners - to gdb as it lists out the whole gdb commands as buttons with a area - for viewing source on which one can include break points, etc by a - click of the mouse, and another area for viewing the debugging results. - - - - DDD: - - The GNU Data Display Debugger, GNU DDD, is a graphical front-end for - command-line debuggers such as GDB, DBX, WDB, Ladebug, JDB, XDB, the - Perl debugger, or the Python debugger. Besides ``usual'' front-end - features such as viewing source texts it also has a good interactive - graphical data display, where data structures are displayed as graphs. - Follow this link for a - DDD manual in postscript / HTML / PDF format. - - - - - - - - Version Control Tools - - It will be worth your while investing some time in learning to use - one of the version control tools below (cvs is what I use ..) if you - are into any serious code development. - - - - - Concurrent Versions System : - CVS is one of the most popular version control systems running on the - Linux operating system. Popular Linux projects like Apache, EGCS, GIMP, - and others are using CVS to coordinate their efforts ... This is how - the URL linked above describes their effort. - - - A tutorial on CVS is available at - Gentoo Linux Documentation and a free CVS book is available - here - - - - - Project Revision Control System : - PRCS, the Project Revision Control System, is the front end to a set of - tools that (like CVS) provide a way to deal with sets of files and - directories as an entity, preserving coherent versions of the entire set. - PRCS was designed primarily by Paul N. Hilfinger, with input and - modifications by Luigi Semenzato and Josh MacDonald. PRCS is written and - maintained by Josh MacDonald. Its purpose is similar to that of SCCS, - RCS, and CVS, but (according to its authors, at least), it is much simpler - than any of those systems. This page is where information on the latest - developments in PRCS can be found. - - - - Gbuild : - gbuild is a script written the Bourne shell language to simplify - package maintenance by allowing you to automate code update from CVS, - compilation, building tar/rpms/srpms of your package. some external - scripts which certain functions of gbuild depend on are written in Perl. - gbuild is released under the GPL. - - - - - - - Integrated Development Environments - - Integrated development environments (IDEs) can be very useful for - building code and ideally come with all the above tools (i.e a compiler, - a debugger and a version control tool). In addition to that IDEs also - usually provide a makefile generator, documenting help, online help - manuals, etc. - - - - Kdeveloper : - A easy to use C/C++ IDE (Integrated Development Environment) for Linux. - It supports KDE/Qt, GNOME, plain C and C++ projects. This site has a lot - of documentation ..... a highly browsable site for software developers. - Specifically, KDevelop manages or provides: - - - All development tools needed for C++ programming like Compiler, - Linker, automake and autoconf; KAppWizard, which generates complete, - ready-to-go sample applications; Class generator, for creating new - classes and integrating them into the current project; File management - for sources, headers, documentation etc. to be included in the - project; The creation of User-Handbooks written with SGML and the - automatic generation of HTML-output with the KDE look and feel; - Automatic HTML-based API-documentation for your project's classes with - cross-references to the used libraries; Internationalization support - for your application, allowing translators to easily add their target - language to a project; - KDevelop also includes WYSIWYG (What you see is what you get)-creation - of user interfaces with a built-in dialog editor; Debugging your - application by integrating KDbg; Editing of project-specific pixmaps - with KIconEdit; The inclusion of any other program you need for - development by adding it to the "Tools"-menu according to your - individual needs. - - - - - VDKbuilder: - VDKbuilder is a tool that helps programmers in constructing GUI interfaces, - editing, compiling, linking, and debugging within an integrated environment. - Using VDKBuilder dramatically reduces developing time since all code related - to GUI construction and signal processing is automatically generated, - maintained and updated. It is distributed under the GNU Public License. - Visit the site for downloading the software. - - - - - - - - Mathematics Packages - - All the links below are free high level languages and Mathematics Packages - for Scientific Computation on Linux. These packages are usually like a - Mathematical Laboratory in which numerical computations can be done and - usually have their own interpreted language. They either link to a - popular (free) plotting package or have their own graphics and plotting - capability. They also provide capability to I/O files and interface - with other programming languages like C, C++, Fortran, etc ... Now a days - some of them have parallel programming capabilities. I have not included - MuPAD, a good symbolic math - package, since is not really free. Check out if their most - free license - suits you. - - - - - Octave: - An excellent package for numerical computations. - It uses gnuplot for plotting - and has a online help. It is also easily extensible (i.e. new functions, - procedures can be written) either using its own language or by using - dynamically loadable modules written in C, C++, Fortran or other languages. - An extensive manual is available - here. - You can get a GNOME based front end for it - here. - It is distributed under the GNU Public License. - - - - - - Scilab: - Another superb package numerical computations having a good user interface - and a very good online click-able help. Its plotting and graphic capabilities - are also very impressive. It also provides for easy interfacing with Fortran - and C. It has its own - free license. - - - - - - Yorick: - Yorick is a fast, interpreted language, designed for scientific computing and - numerical analysis. The syntax is similar to C, but the variables need not - be declared. It offers an interactive graphics package based on X windows. - X-Y plots, quadrilateral meshes, filled meshes, cell arrays, and contours are - supported. You can embed compiled routines in Yorick to solve problems for - which the interpreter is too slow. It is also useful as a pre and post - processor for large physical simulation programs. A tutorial like manual - is available - - here. - Yorick is open source software, - - copyright of the Regents of the University of California. - - - - - - Algae: - As the above link describes it, Algae is a interpreted language for - numerical analysis. It was developed as a fast and versatile - tool, capable of handling large problems. Algae consists of the programming - language Algae, and algae, the interpreter. - Its features include speed (generally much faster than octave, RLaB - and Scilab), storage of sparse arrays and a code profiling capability - (to check where your code spends its time). A user manual is available - here. - It is distributed under the GNU General Public License. - - - - - YACAS: - As the above link describes it, "YACAS is an easy to use, general purpose - Computer Algebra System, a program for symbolic manipulation of - mathematical expressions. It uses its own programming language designed for - symbolic as well as arbitrary-precision numerical computations". Links to - documentation (user manual, tutorial, etc ..) is available - here. - It is distributed under the GNU General Public License. - - - - - RLAB: - The above link describes it thus, "Rlab is an interactive, interpreted - scientific programming environment. Rlab is a very high level language - intended to provide fast prototyping and program development, as well - as easy data-visualization, and processing". It is distributed under the - GNU General Public License. The author Ian Searle has written an article in - The Linux Journal titled - An Introduction to Rlab - - which as he reminds us, is a bit dated, and a - - Reference Manual is also available. - - - - - Maxima: - Maxima is a symbolic computation program. - As the link above describes it, "Maxima is a Common Lisp implementation of - MIT's Macsyma system for computer based algebra". - George White has maintained and extended Maxima for the last 18 years, - but has formal permission from DOE, to release this under GPL as a derivative - work science 1998. I am looking forward to installing this and yacas above - to check out their symbolic computation capabilities. - - - - -The R-Project for Statistical Computing: -R is a language and environment for statistical computing and graphics. -It provides a large collection of tools for statistical analysis of large -arrays of data and also graphical facilities. R is also a complete effective -programming language. For computationally intensive tasks, C, C++ and Fortran -code can be linked and called at run time. A comprehensive set of manuals -dealing with installation, introduction, writing extensions, etc ... is -available here. -It is distributed under the GNU General Public License. - - - - -gTybalt: -gTybalt is a step towards a free computer algebra system. It is object -oriented, allowing symbolic calculations within C++. It is efficient, -in the sense that solutions developed with gTybalt can be compiled with -a C++ compiler and executed independently of gTybalt. The mathematical -formulae are visualized using TeX fonts and can easily be converted to -LaTeX. I did not realize that it has good graphic capabilities till I -checked out the gTybalt -manual. -It is distributed under the GNU General Public License. - - - - - JACAL: -As the link above describes it, " JACAL is an interactive symbolic mathematics -program. JACAL can manipulate and simplify equations, scalars, vectors, -and matrices of single and multiple valued algebraic expressions containing -numbers, variables, radicals, and algebraic differential, and holonomic -functions". - - - - - bc: -bc is an arbitrary precision numeric processing language. It supports -interactive execution of statements. Click here for a Manual in a variety of formats. -It is GNU software and is distributed under the GNU General Public License. - - - - - - Numerical Methods and Libraries - - The best thing that could happen for scientific computing with free - software on GNU/Linux is the GNU Scientific Library - GSL. - It however has source code only in C and people who use FORTRAN - will find that a let down. Pouncing on this opportunity it is - recommended that GSL is another reason (in addition to the GCC - C compiler, coupled with the advantages of C programming) - for starting to learn to use C. In addition to this, - the two best source code repositories for Numerical Methods and - libraries are Netlib - and GAMS. - There are new numerical packages being developed outside the usual - "write a FORTRAN program, get a numerical subroutine from INTERNET - for solving the numerics" concepts. The merits and demerits of this - approach are debatable, but there exist more options like - Object Oriented Numerics - GSL and GiNaC which are - exciting developments. - - - - Repositories - - - - - Netlib: - An amazing amount of free source code for Numerical Methods. Netlib - is THE source code repository which contains an innumerable - amount of source code for Numerical Methods. It also has an active - discussion forum - wherein you can submit your queries and stay posted for - help. Netlib also has a - Parallel Tools Library and a search by subject. - - - - - - - GAMS: Guide to Available Mathematical Software - GAMS has a very useful search using which one can search for - keywords (example: ``diffusion'' to search for a diffusion equation - solver). However the browse by package at GAMS reveals that a lot of - the software they provide is a link to the netlib repository. - - - - - Object Oriented Numerics - A site devoted to object oriented numerics. It has a Mailing list, - Extensive Links to freely available libraries (OO of course) - and freely available tools for object oriented scientific computing. - - - - - GNU Scientific Library - The GNU Scientific Library (GSL) is a collection of numerical routines - written from scratch in C. It provides an Applications Programming - Interface (API) for C programmers and also allows wrappers to be written - for very high level languages. It covers a wide range of numerical - computing topics, has a good manual, is widely portable and is distributed - under the GNU General Public License. - - - - - GiNaC - GiNaC is designed to allow the creation of software which need symbolic - manipulations embedded in them. It extends C++ by a set of algebraic - capabilities and is recursively named for GiNaC is not a Computer Algebra - system. It is distributed under the terms and conditions of the GNU - general public license (GPL). - - - - - - - Other topic specific numerical libraries - - - - FFTW - FFTW is a collection of fast C routines for computing the Discrete - Fourier Transform in one or more dimensions. It includes complex, - real, and parallel transforms, and can handle arbitrary array sizes - efficiently. This package includes both the double- and - single-precision FFTW uniprocessors and the threads libraries. - - - - - - LAPACK - LAPACK (Linear Algebra PACKage) is a standard library for numerical - linear algebra. LAPACK provides routines for solving systems of - simultaneous linear equations, least-squares solutions of linear - systems of equations, eigenvalue problems, and singular value - problems. LAPACK is coded in Fortran77 and is built with egcs. - It is well documented and widely used (and therefore widely tested). - - - - - - SuperLU - SuperLU is a general purpose library which performs an LU decomposition - for the direct solution of large, sparse, non-symmetric systems of linear - equations on high performance machines. Its written in C and is callable - from either C or Fortran. - - - - - - ARPACK - - ARPACK is a set of Fortran77 subroutines designed to solve large scale - eigenvalue problems. A Users Guide - is available. The above link also gives information about a - parallel version of ARPACK - PARPACK and a object oriented version - ARPACK++. - - - - - - - Computational Fluid Dynamics codes - This link contains a comprehensive listing of public domain, shareware - and freeware Computational Fluid Dynamics codes links with a description - of each CFD code. - - - - - - - - - Graphics and Visualization - - - - Gnuplot - Gnuplot is a command-line driven interactive function plotting - utility. It handles both curves (2 dimensions) and surfaces (3 - dimensions). Surfaces can be floating in the 3-d coordinate space, - or as a contour plot. For 2-d plots, there are also many plot - styles, including lines, points, lines with points, error bars, - and impulses. Graphs may be labeled with arbitrary labels and - arrows, axes labels, a title, date and time, and a key. It has - multiple plotting capabilities too. It allows saving the graphs in - various formats which can be included in word processors. - It can be used to generate publication quality plots. - - - - - - NCAR Graphics - A very popular graphics package which is very well documented and widely - used. It provides basic ingredients for creating complex plots as - functions / routines that can be called from Fortran and C. There is - a contributed programming interface to the NCAR Graphics package: - NCL (NCAR Command Language). The programming interfaces provide access - to complex graphics utilities like contouring, world map projections, - and velocity vectors. For the most part, the C interface is built on - top of the Fortran interface... It is distributed under the GNU public - license. Click here - for going to the documentation of all its various components. - - - - - OpenDX - A very good Open Source Data eXplorer. It can handle large amounts of data - and creates great visualizations. It was the tool I stumbled upon when - I wanted a free graphics routine to make 3-D plots and zoom-in, rotate, - and really eXplore the output Data from my codes. The downside is that - compiling from source is really challenging and getting started is a - difficult. However it has excellent documentation distributed with it - and once I started off it was the best tool I have ever used. - - - - - Gri: - It is a language for scientific graphics programming. The claim that Gri - is similar to LaTeX in the sense that both provide extensive power as a - reward for tolerating a learning curve seems exciting and I for one want - to check this out!! Check out the following - article - in The Linux Journal. Go to the gri home page if you are now impressed by - it and check out download info and manuals. - - - - - MayaVi: - A scientific data visualizer written in Python. It is distributed under the - BSD - license. The screenshots look promising. Check out the above link - for more details. - - - - - PGPLOT: - PGPLOT is a Fortran 77 or C callable subroutine package for drawing - scientific 2D and Simple 3D plots. One can call these routines during - runtime and redirect the output to a variety of devices at run time. - It is well documented and the full documentation is available at the - above site. It is Free for Non-Commercial Use. A user manual is available - online at PGPLOT - Users Manual - - - - - PLplot: - This is a library of scientific plotting functions that can be called from - C, C++, FORTRAN, TCL, PYTHON. PLplot features as described in the above link - are, "It can be used to create standard x-y plots, semilog plots, - log-log plots, contour plots, 3D plots, mesh plots, bar charts and pie charts. - Multiple graphs (of the same or different sizes) may be placed on a single - page with multiple lines in each graph. There are almost 2000 characters - in the extended character set. This includes four different fonts, the - Greek alphabet and a host of mathematical, musical, and other symbols. - A variety of output devices are supported and new devices can be easily - added by writing a small number of device dependent routines". To download - click - - here . - - - - - Grace - Grace is a WYSIWYG 2D plotting tool for the X Window System and Motif. - Grace runs on practically any version of Unix. Grace is a descendant of - ACE/gr, also known as Xmgr. It is lisenced under the GNU public license. - This link also has a tutorial and download information. - - - - - SciGraphica - - SciGraphica is a application for data analysis and technical graphics. - It fully supplies plotting features for 2D, 3D and polar charts. The - aim is to obtain a fully-featured, cross-platform, user-friendly, - self-growing scientific application. It is free and open-source, - released under the GPL license. - - - - - - Plotutils: - The GNU plotutils package contains software for both programmers and - technical users. Its centerpiece is libplot.a powerful C/C++ function - library for exporting 2-D vector graphics in many file formats, both - vector and raster. It can also do vector graphics animations. - Besides libplot, the package contains command-line programs for - plotting scientific data. Many of them use libplot to export graphics. - - - - - DISLIN - DISLIN is a high-level plotting library for displaying data as curves, - polar plots, bar graphs, pie charts, 3D-color plots, surfaces, contours - and maps. - - - - - ImLib3D - ImLib3D is an open source C++ library for 3D (volumetric) image processing. - It contains most basic image processing algorithms, and some more - sophisticated ones. It comes with an optional viewer that features - multi-planar views, animations, vector field views and 3D (OpenGL) - multi-planar. - - - - - Ptplot: - Ptplot is a 2D data plotter and histogram tool implemented in Java. - Ptplot can be used as a standalone applet or application, or it can be - embedded in your own applet or application. - - - - - - Programming systems for GNU/Linux - - This section deals with links to tutorials and documents for installing - Linux on a PC, getting started with Linux, and then going a step further - -- to optimize your PC for processing power, using multiple processors - (Symmetric Muliti Processing - SMP); making a cheap, upgradeable - Supercomputing Linux cluster and finally links to software to do parallel - programming on Linux. - - - The GNU/Linux Workstation - - As with most documentation related to GNU/Linux, - the Linux Documentation project's - home page is a priceless source. You might first want to read - - The Linux Installation HOWTO. For those who want to install - Linux along with Windows might want to browse through - - The Linux + Windows HOWTO. When installing Linux make sure - that you choose to install all documentation. After installing Linux, - a good, comprehensive document to getting started with using Linux is - The Rute Users Tutorial and - Exposition which is a beginners guide to Linux and UNIX like - systems. I'd like to give a less intimidating (size-wise) link to a - small beginners guide, but U will find this useful after taking the - plunge. You might also want to go through The Linux System - Administrator's Guide and to check out - The Linux Administration Made Easy (LAME) guide - It attempts to describe day-to-day administration and maintenance - issues commonly faced by Linux system administrators. - - - - Parallel Processing and Symmetric Multiprocessing: - Supercomputing - - It is possible to get large volume number crunching without - spending millions of rupees on a supercomputer. You only need - to link together (by some high speed network) the requisite - number of CPUs, with GNU/LINUX as the underlying OS. Add - some freely available message passing software and a effective - parallel processing number crunching machine is made. Such - clusters are called "Beowulf clusters". The other advantages - of such a cluster other than building costs is, up-gradation - costs are minimal. The two best resources for Linux cluster - builders are - - - The Beowulf - Project home page and - - - The Extreme - Linux Project - - - - These sites are upgraded frequently with useful information - for cluster builders. - - - Parallel computing document links - - You will also want to read this excellent article on - Linux - Clustering Software (and the large variety of links - it provides) by Joe Greenseid. I hope to go through the links - and include them subsequently in this HOWTO. - - - - Other free document links for parallel processing are: - - - - - - The Beowulf Howto : - This document introduces the Beowulf Supercomputer architecture - and provides background information on parallel programming, - including links to other more specific documents, and web pages. - But, before that for an understanding of parallel processing and - Symmetric multiprocessing on Linux, check out the following: - - - - - - The Parallel Processing on Linux HOWTO - - - - - - The Symmetric Multiprocessing HOWTO - - - - - - - Parallel processing software for Linux - - Now after reading the above documents, you have an idea of parallel - processing. Parallel program libraries are the core of parallel processing - on a Linux cluster. There are various free implementations of parallel - processing libraries. Since parallel processing is all about performance, - these libraries have some very nice functional tools to analyze your parallel - program performance. Given below is a set of links to these parallel - program libraries and tools. - - - - - - - - Message Passing Interface: - MPI is a standard specification of message passing libraries. The above - document gives a lot of links to documents on the standard, etc.. A MPI - implementation for Linux - mpich is also - available at that site. There are a lot of documents for - Learning to use MPI - . - - - - - - Local Area Multicomputer - - LAM: - LAM (Local Area Multicomputer) is an MPI programming environment and - development system for heterogeneous computers on a network. - With LAM, a dedicated cluster or an existing network computing - infrastructure can act as one parallel computer solving one problem. - LAM features extensive debugging support in the application development - cycle and peak performance for production applications. LAM features a - full implementation of the MPI communication standard. - You can download the sources (tar-zipped, rpm) or binaries from - here - A host of MPI tutorial links and also a `getting started with LAM' - tutorial is available - here - - - - - - - Parallel Virtual Machine : - As the PVM home page describes, it is a software package that permits - a heterogeneous collection of Unix and/or NT computers hooked together - by a network to be used as a single large parallel computer. Thus large - computational problems can be solved more cost effectively by using the - aggregate power and memory of many computers. The software is very - portable. The source, which is available free thru netlib, has been - compiled on everything from laptops to CRAYs. - - - - - - Ganglia: - Ganglia is an open source cluster monitoring and execution environment - developed at the University of California, Berkeley Computer Science - Division. As the above link describes it, "Ganglia is as simple to - install and use on a 16-node cluster as it is to use on a 512-node - cluster as has been proven by its use on multiple 500+ node clusters". - It not only can link nodes in a cluster, but also link clusters to other - clusters. - - - - - - - - - - Word-Processing and Poster presenting tools on Linux - - - Those of you who do not use - LaTeX and find it challenging and want a WYSIWYG word processor, - keep your ears tuned to OpenOffice - - which has released version 1.2 of its openoffice software. Its tools - may compare well with the best in the market. - - - - Word Processing Tools - - - - - Latex: - LaTeX is a high-quality typesetting system, with features designed for - the production of technical and scientific documentation. LaTeX is the - de facto standard for the communication and publication of scientific - documents. - - David R.Wilkin's primer "Getting Started with LaTeX" - is a good tutorial to getting started with LaTeX. - For those who have to live with a WYSIWYG documenting tool, check - LyX. This is a front-end for - latex. It isn't as powerful as latex proper, but helps with a good WYSIWIG. - - - - - - Abiword: - As the AbiWord home page says, - "AbiWord is suitable for typing papers, letters, reports, memos, and so forth". - It has won many awards and seems to be the best open source WYSIWYG word - processor. Check out the above link to know more about it and download it. - - - - - - kword: - As the kword home page says, - "KWord is a FrameMaker-like word-processing and desktop publishing application. - KWord is capable of creating demanding and professional looking documents. - Whether you are a corporate or home user, production artist or student, KWord - will prove a valuable and easy to use tool for all your word processing and - layout needs". Check out the above link to know more about it and download it. - (you might want to know more about the whole - koffice suite). - - - - - - - Poster presentation tools - - - - - KPresenter: - KPresenter is the presentation tool of the KOffice suite of office - utilities. It allows screen presentations with all the trappings one - is used to seeing in costly presentation tools. It also allows honest, - real scientific presentations where one does not have to impress the - audience with non subject specific stuff :-). The best thing about it - is the possibility of saving the presentation as a html file. It makes - portable network graphics files with each presentation slide. With a - smattering of knowledge of html files one could put in a animated image - as a image link thereby allowing one to show movies too when necessary. - - - - - - Xfig : - Though the man page claims that it is a facility for the - interactive generation of figures ...., It in fact much more than that. - Other than generating figures for elucidating what you want to say in - a poster, it helps you import and export figures in a variety of formats, - write text in various fonts and sizes, generate Greek symbols and color - text, Save as latex picture file or any other format supported by your - word processor for inclusion in your publications, generate GIFs of each - page of the poster to put on your web site, and finally it generates - *.fig files which are small in size. The only thing on my wish list for - xfig is the capability to edit the imported pictures which are not in - *.fig format. Therefore for a computer screen projected poster - presentation you need a frames capable browser with contents in one frame - and the xfig generated posters (exported as *.png or *.jpg from Xfig) on - the other. - - - - - - - - Free Database Management Systems for Linux - - Scientific computing has two parallel data needs, one the physical values - of the data itself, and the other is Database systems to manage the data. - In this document links are provided only to database resources on the net - and free Database systems. I personally do not use databases to manipulate - the data generated by my codes. gawk, sed, and other basic Unix commands - like grep, head and tail seem sufficient to manipulate any data I generate. - I thought I should include this section for the large data generators. - Hopefully a person with experience in databases will make this section better. - - - - - - - - Free database list - - - - - - - ACM SIGMOD: - Index of publicly available database software. - - - - - - MySQL: - A relational Database management system. - - - - - - PostgreSQL - As the link above describes it ...PostgreSQL is - a sophisticated Object-Relational DBMS, supporting almost all SQL - constructs, including subselects, transactions, and user-defined types - - - - - - - - Linux in the laboratory - - Again this is a section where I have zero experience and hope someone will - with experience will contribute towards making this document better. However, - I provide below links suggested by Sambaran Pahari and Deepak Gupta. These - links seem to be very good from my inexperienced viewpoint. - - - - - The Linux Lab Project - A site for "Linux Lab Project."..everything to do with laboratory process, - process control, automation and data acquisition on Linux. - As the above link says, "The Linux lab project is intended to help people with - development of data collection and process control software for LINUX. - It is planned to provide a standardized development environment for a wide variety - of applications from hardware support to application development". - - - - - - Linux Parallel port drivers: - The above link says, "If you have a parallel port device and would - like to know if there is a Linux driver available for it --then this is - the place to look". Sounds like a confident claim. - - - - -
- + + + +
+ + + + Scientific Computing with Free software on GNU/Linux HOWTO + + + Manoj + Warrier + + +
m_war (at) users.sourceforge.net
+ + + + Shishir + Deshpande + + +
shishir (at) ipr.res.in
+ + + + V. S. + Ashoka + + +
ashok (at) rri.res.in
+ + + 2003-10-03 + + + + + + 1.1 + 2004-06-21 + M. W + Updates and evaluated distros + + + + 1.0 + 2003-11-18 + JP + Document Reviewed by LDP. + + + + 0.0 + 2003-10-01 + M. W + first draft proposed + + + + + + + + + This document aims to show how a PC running GNU/Linux can be used + for scientific computing. It lists the various available free software + and also links on the world wide web to tutorials on getting started + with the tools. + + + + + + + Preamble + + + + Copyright and License + + + + This document, Scientific Computing with free software on GNU/Linux HOWTO, + is copyrighted (c) 2002 by Manoj Warrier. + Permission is granted to copy, distribute and/or modify this document under the terms of the + GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; + with no Invariant Sections, + with no Front-Cover Texts, and with no Back-Cover Texts. + A copy of the license is available + + here + + + + + Disclaimer + + + No liability for the contents of this document is accepted. + Use of the concepts, examples, links and information is entirely at + your own risk. + There may be errors and inaccuracies, that could damage your system, + waste your time, etc... Proceed with caution, and although + this is unlikely, the author takes no responsibility whatsoever. + + + + All copyrights are held by their respective owners, unless + specifically noted otherwise. Use of a term in this document + should not be regarded as affecting the validity of any trademark + or service mark. Naming of particular products, software or + brands should not be seen as endorsements + + + + I have not used many of the software applications to which links are provided. + There are simply too many applications that do the same thing, + that one cannot be expected to have used all of them. In a book + on Scientific Computing using GNU/Linux, one would + mention ones favorite tool to carry out a task and describe it in + detail. However this is a howto providing links to various + available free tools for scientific computing and may contain + links to some software that promises much but delivers little + and vice versa. + + + + + + Motivation + + This howto mainly consists of the links provided at + http://Scilinux.sourceforge.net which has to be disbanded due to a name + conflict. The best alternative seems to be to make it a Linux document + and host it at the LDP site. Another reason is that there seems to be + many free software applications doing the same things. We hope to provide links to + the available software thereby making it easy for the scientific community + to make a choice without spending much time. + + + + + + Credits / Contributors + + + In this document, I have the pleasure of acknowledging: + + + + + + Linus Trovalds, Richard M. Stallman and their merry men for + Linux, GNU and also for indirectly broadening various + perspectives which were not really obvious. + + + + + A host of colleagues and friends from the + Institute for Plasma Research, India + for discussions at various times. + + + + + Marcel Bose, Ivan Lamouret, K. Scott Hunziker, Livine Christin, + W. Herbert, Simon Pinches and many others for suggesting various + links mentioned in this document. + + + + + Vasudha my wife for letting me do what I wish and egging me on + with comments like "let us hope that you will finish at least + this project" + + + + + Shishir and Ashoka are co-authors of this document because such a + collection of links was Shishir's idea and Ashoka is always + contributing by providing links, suggestions and a second point + of view. They will be helping me maintain this HOWTO too. + + + + + + Feedback + + Feedback is most certainly welcome for this document. Send + your additions, comments and criticisms to the following + email address : m_war at users.sourceforge.net. + + + + + + Translations + + No translations yet. + + + + + + + + + Introduction + + GNU/Linux is probably the platform of choice for scientific + computing. There exists a wide variety of high level languages, + debugging tools and other code development tools for + programming, numerical subroutines for solving various types of + equations, plotting and visualization packages, word processing + software which can display equations and figures and in fact parallel + programming software to construct a supercomputer with off the shelf + PC parts and some hardware. This document aims to provide a list + of free software for carrying out the above + tasks and links to tutorials and other documents on how to setup and + use these software applications. + + + + This document does not aim to provide links to subject specific free + software available for GNU/Linux systems. It aims to show how + GNU/Linux can be used best to handle scientific computing tasks. + It is hoped that people or institutions with interest in a + specific subject list, compile a list of the free software available + for that subject ... for example see Linux for Astronomy, + Linux for Biotechnology and Linux for + Chemistry at + The Random Factory + . Another site with a lot of links (to commercial and + free) scientific software is + Scientific Applications + on Linux. The + GNU Software Directory also has links to many of the links + provided in this howto plus many more topic specific software. + You may also want to check out + The Science and Engineering section at Freshmeat.net. + + + + The software links provided are classified into + + + Code development tools + + + Mathematics packages + + + Numerical subroutines and libraries + + + Graphics and visualization + + + GNU/Linux Systems + + + Publishing tools + + + Databases + + + Linux in the Laboratory + + + + + + Just installing GNU/Linux on your PC makes it a powerful workstation. + The various popular distributions however do not have all the tools + needed to make it the ideal scientific computing machine. This HOWTO + aims to fill in this gap by creating a list of free software useful + for scientific computing. It is assumed that people reading this + document already have a PC with Linux and the GNU utilities installed. + For those who do not have such a setup and want to install GNU-Linux + can check out GNU/Linux Systems + for links to documents on installing GNU/Linux, and also on how to get + started using GNU/Linux. Recently there has been an effort by Dirk + Eddelbuettel to create a scientific computing environment + Quantian + which probably is the first GNU-Linux distribution tailored for + Scientists. I checked out the latest release and it has almost + all the packages mentioned in this document and many packages + not mentioned. It is fair to say that if you have any linux distribution + in which the packages are managed by rpms or any debian based system, + you will find pre-compiled binaries of these packages and will not + have to waste much time installing them. + + + + + Code Development Tools + + Code development consists of mainly Programming languages, + Debugging tools, Version Management tools, Compiling tools, + and Integrated Development Environments (IDEs) where all + the above are coupled as a single software application. + + + + + Programming Languages + + Links are provided to various compilers used in Scientific + Computing like FORTRAN, C, C++, Java and more recently + Python. + + + + + + GNU Compiler Collection + : + GNU's project to produce a world class optimizing compiler. + It works on multiple architectures and diverse environments. + Currently GCC contains front ends for C, C++, Objective C, + GNU Fortran-95, Java, and Ada, as well as libraries for these languages + (libstdc++, libgcj,..). + + + For manuals on using the various GCC compilers check out + + The GCC online documentation + + + + + + g77 + : + The GCC front end for FORTRAN 77. It is a very good FORTRAN77 + compiler. It however does not have the -r8 option which compiles + a program as double precision. This could be a good compiler + design philosophy but in many cases gives problems when porting + a code from SUN / DEC / HP workstations onto Linux systems. + The g77 manual is available at + The Gcc Online + documentation site. + + + + + + gfortran. + I was happy to receive this link by mail. It was 3 years since I had + migrated to using the GNU C compiler for scientific computing because + there was no "truly free" FORTRAN-95 compiler available then. I thank + Paul Thomas for this link. + + + + + + g95. + gfortran above and g95 are reportedly offshoots from the same CVS tree. + Has an impressive list of programs that compiles and runs using this + compiler. + + + + + + + fort77 and f2c: + fort77 is a perl program which invokes the f2c command + (a Fortran to C translator) transparently, so it can + be used just like a real Fortran compiler. Fort77 can + be used to compile Fortran, C and assembler code and + can link the code with f2c libraries. If you install + fort77, you'll also need to install the f2c package. + This does not have the "-r8" problem. You can download + fort77 and f2c from the above link. + + + + + + lush: + An object-oriented programming language, which combines the flexibility + of an interpretive language, with the efficiency of a compiled language. + It has full interfaces to numerical libraries (GSL, LAPACK, BLAS), + graphics libraries (OpenGL), which allows creation of graphics and 3D + animations and many other features that sound too good to be + true. I have not yet tried this out, but it sounds very promising. + + + + + + Scientific Python: + You may want to explore + Python for your + scientific computing needs. Python is an interpreted, + interactive, object-oriented programming language. + + It has a number of extensions for numerics, plotting, + data storage and combined with Tk lets you develop very + good GUIs for your codes. The most exciting aspect is that + it simplifies programming because it has modules for almost + anything (vectors, tensors, transformations, derivatives, + linear algebra, Fourier transforms, statistics, etc ...) are + available. You can also wrap C and Fortran libraries from + Python. Finally if you want to write a numerical scheme of + your own you may find that it is simpler in Python. + There are also interfaces to netCDF (portable binary files), + MPI and BSPlib (parallel programming). + + + You can further explore Python for Scientific computing here: + + + + + + Scientific-Python: + A collection of modules for scientific computing on Python. + All the necessary modules can be downloaded as either a tar file + or an RPM file from here. The maintainer Konrad HINSEN also has + a nice tutorial on + + Scientific Computing in Python. + + + + + SciPy An open source + library of scientific tools for Python. It includes modules for + graphics and plotting, optimization, integration, special + functions, signal and image processing, genetic algorithms, + ODE solvers, etc. + + + + + + + + + + + Debugging Tools + + In this section links are given to mainly debugging tools for GCC and + FORTRAN. I understand that python has a debugging module built in though I + have not used it. The purpose of a debugger is to allow you to see what is + going on inside a program while it executes or what the program was doing + when/if it crashed. + + + + Ftnchek: + A FORTRAN checker designed to detect errors in a Fortran + program that a compiler usually does not. Therefore it is best + to run ftnchek on your FORTRAN programs after it has compiled + without errors. Its purpose is to assist the user in finding + semantic errors. Semantic errors are legal in the Fortran + language but are wasteful or may cause incorrect operation. + An on-line + manual is available. This project is looking for volunteers + to bringing ftnchek up to the Fortran 90 standard. + + + + gdb + : + All programs written in the languages supported by GCC can be + debugged using gdb, an excellent interactive, + command line debugger. You can compile your programs using a -g option + which then compiles your code with debugging information inserted + into the executable. It can start your programs, stop your programs + on specified conditions and at specified locations, examine what + happened when your program stops. In a large code with multiple + cascading calls to various functions it can back trace the function + calls. You can also + Download + the document Debugging with GDB and + a quick reference card. + + + + xxgdb: + It is a front end to the gdb debugger. Useful for beginners + to gdb as it lists out the whole gdb commands as buttons with a area + for viewing source on which one can include break points, etc by a + click of the mouse, and another area for viewing the debugging results. + + + + DDD: + + The GNU Data Display Debugger, GNU DDD, is a graphical front-end for + command-line debuggers such as GDB, DBX, WDB, Ladebug, JDB, XDB, the + Perl debugger, or the Python debugger. Besides ``usual'' front-end + features such as viewing source texts it also has a good interactive + graphical data display, where data structures are displayed as graphs. + Follow this link for a + DDD manual in postscript / HTML / PDF format. + + + + + + + + Version Control Tools + + It will be worth your while investing some time in learning to use + one of the version control tools below (cvs is what I use ..) if you + are into any serious code development. + + + + + Concurrent Versions System : + CVS is one of the most popular version control systems running on the + Linux operating system. Popular Linux projects like Apache, EGCS, GIMP, + and others are using CVS to coordinate their efforts ... This is how + the URL linked above describes their effort. + + + A tutorial on CVS is available at + Gentoo Linux Documentation and a free CVS book is available + here + + + + + Project Revision Control System : + PRCS, the Project Revision Control System, is the front end to a set of + tools that (like CVS) provide a way to deal with sets of files and + directories as an entity, preserving coherent versions of the entire set. + PRCS was designed primarily by Paul N. Hilfinger, with input and + modifications by Luigi Semenzato and Josh MacDonald. PRCS is written and + maintained by Josh MacDonald. Its purpose is similar to that of SCCS, + RCS, and CVS, but (according to its authors, at least), it is much simpler + than any of those systems. This page is where information on the latest + developments in PRCS can be found. + + + + Gbuild : + gbuild is a script written in the Bourne shell language to simplify + package maintenance by allowing you to automate code update from CVS, + compilation, building tar/rpms/srpms of your package. some external + scripts which certain functions of gbuild depend on are written in Perl. + gbuild is released under the GPL. + + + + + + + Integrated Development Environments + + Integrated development environments (IDEs) can be very useful for + building code and ideally come with all the above tools (i.e a compiler, + a debugger and a version control tool). In addition to that IDEs also + usually provide a makefile generator, documenting help, online help + manuals, etc. + + + + Kdeveloper : + A easy to use C/C++ IDE (Integrated Development Environment) for Linux. + It supports KDE/Qt, GNOME, plain C and C++ projects. This site has a lot + of documentation ..... a highly browsable site for software developers. + Specifically, KDevelop manages or provides: + + + All development tools needed for C++ programming like Compiler, + Linker, automake and autoconf; KAppWizard, which generates complete, + ready-to-go sample applications; Class generator, for creating new + classes and integrating them into the current project; File management + for sources, headers, documentation etc. to be included in the + project; The creation of User-Handbooks written with SGML and the + automatic generation of HTML-output with the KDE look and feel; + Automatic HTML-based API-documentation for your project's classes with + cross-references to the used libraries; Internationalization support + for your application, allowing translators to easily add their target + language to a project; + KDevelop also includes WYSIWYG (What you see is what you get)-creation + of user interfaces with a built-in dialog editor; Debugging your + application by integrating KDbg; Editing of project-specific pixmaps + with KIconEdit; The inclusion of any other program you need for + development by adding it to the "Tools"-menu according to your + individual needs. + + + + + VDKbuilder: + VDKbuilder is a tool that helps programmers in constructing GUI interfaces, + editing, compiling, linking, and debugging within an integrated environment. + Using VDKBuilder dramatically reduces developing time since all code related + to GUI construction and signal processing is automatically generated, + maintained and updated. It is distributed under the GNU Public License. + Visit the site for downloading the software. + + + + + + + + Mathematics Packages + + All the links below are free high level languages and Mathematics Packages + for Scientific Computation on Linux. These packages are usually like a + Mathematical Laboratory in which numerical computations can be done and + usually have their own interpreted language. They either link to a + popular (free) plotting package or have their own graphics and plotting + capability. They also provide capability to I/O files and interface + with other programming languages like C, C++, Fortran, etc ... Now a days + some of them have parallel programming capabilities. I have not included + MuPAD, a good symbolic math + package, since is not really free. Check out if their most + free license + suits you. + + + + + Octave: + An excellent package for numerical computations. + It uses gnuplot for plotting + and has a online help. It is also easily extensible (i.e. new functions, + procedures can be written) either using its own language or by using + dynamically loadable modules written in C, C++, Fortran or other languages. + An extensive manual is available + here. + You can get a GNOME based front end for it + here. + It is distributed under the GNU Public License. + + + + + + Scilab: + Another superb package numerical computations having a good user interface + and a very good online click-able help. Its plotting and graphic capabilities + are also very impressive. It also provides for easy interfacing with Fortran + and C. It has its own + free license. + + + + + + Yorick: + Yorick is a fast, interpreted language, designed for scientific computing and + numerical analysis. The syntax is similar to C, but the variables need not + be declared. It offers an interactive graphics package based on X windows. + X-Y plots, quadrilateral meshes, filled meshes, cell arrays, and contours are + supported. You can embed compiled routines in Yorick to solve problems for + which the interpreter is too slow. It is also useful as a pre and post + processor for large physical simulation programs. A tutorial like manual + is available + + here. + Yorick is open source software, + + copyright of the Regents of the University of California. + + + + + + Algae: + As the above link describes it, Algae is a interpreted language for + numerical analysis. It was developed as a fast and versatile + tool, capable of handling large problems. Algae consists of the programming + language Algae, and algae, the interpreter. + Its features include speed (generally much faster than octave, RLaB + and Scilab), storage of sparse arrays and a code profiling capability + (to check where your code spends its time). A user manual is available + here. + It is distributed under the GNU General Public License. + + + + + YACAS: + As the above link describes it, "YACAS is an easy to use, general purpose + Computer Algebra System, a program for symbolic manipulation of + mathematical expressions. It uses its own programming language designed for + symbolic as well as arbitrary-precision numerical computations". Links to + documentation (user manual, tutorial, etc ..) is available + here. + It is distributed under the GNU General Public License. + + + + + RLAB: + The above link describes it thus, "Rlab is an interactive, interpreted + scientific programming environment. Rlab is a very high level language + intended to provide fast prototyping and program development, as well + as easy data-visualization, and processing". It is distributed under the + GNU General Public License. The author Ian Searle has written an article in + The Linux Journal titled + An Introduction to Rlab + + which as he reminds us, is a bit dated, and a + + Reference Manual is also available. + + + + + Maxima: + Maxima is a symbolic computation program. + As the link above describes it, "Maxima is a Common Lisp implementation of + MIT's Macsyma system for computer based algebra". + George White has maintained and extended Maxima for the last 18 years, + but has formal permission from DOE, to release this under GPL as a derivative + work science 1998. I am looking forward to installing this and yacas above + to check out their symbolic computation capabilities. + + + + +The R-Project for Statistical Computing: +R is a language and environment for statistical computing and graphics. +It provides a large collection of tools for statistical analysis of large +arrays of data and also graphical facilities. R is also a complete effective +programming language. For computationally intensive tasks, C, C++ and Fortran +code can be linked and called at run time. A comprehensive set of manuals +dealing with installation, introduction, writing extensions, etc ... is +available here. +It is distributed under the GNU General Public License. + + + + +gTybalt: +gTybalt is a step towards a free computer algebra system. It is object +oriented, allowing symbolic calculations within C++. It is efficient, +in the sense that solutions developed with gTybalt can be compiled with +a C++ compiler and executed independently of gTybalt. The mathematical +formulae are visualized using TeX fonts and can easily be converted to +LaTeX. I did not realize that it has good graphic capabilities till I +checked out the gTybalt +manual. +It is distributed under the GNU General Public License. + + + + + JACAL: +As the link above describes it, " JACAL is an interactive symbolic mathematics +program. JACAL can manipulate and simplify equations, scalars, vectors, +and matrices of single and multiple valued algebraic expressions containing +numbers, variables, radicals, and algebraic differential, and holonomic +functions". + + + + + bc: +bc is an arbitrary precision numeric processing language. It supports +interactive execution of statements. Click here for a Manual in a variety of formats. +It is GNU software and is distributed under the GNU General Public License. + + + + + + Numerical Methods and Libraries + + The best thing that could happen for scientific computing with free + software on GNU/Linux is the GNU Scientific Library + GSL. + It however has source code only in C and people who use FORTRAN + will find that a let down. Pouncing on this opportunity it is + recommended that GSL is another reason (in addition to the GCC + C compiler, coupled with the advantages of C programming) + for starting to learn to use C. In addition to this, + the two best source code repositories for Numerical Methods and + libraries are Netlib + and GAMS. + There are new numerical packages being developed outside the usual + "write a FORTRAN program, get a numerical subroutine from INTERNET + for solving the numerics" concepts. The merits and demerits of this + approach are debatable, but there exist more options like + Object Oriented Numerics + GSL and GiNaC which are + exciting developments. + + + + Repositories + + + + + Netlib: + An amazing amount of free source code for Numerical Methods. Netlib + is THE source code repository which contains an innumerable + amount of source code for Numerical Methods. It also has an active + discussion forum + wherein you can submit your queries and stay posted for + help. Netlib also has a + Parallel Tools Library and a search by subject. + + + + + + + GAMS: Guide to Available Mathematical Software + GAMS has a very useful search using which one can search for + keywords (example: ``diffusion'' to search for a diffusion equation + solver). However the browse by package at GAMS reveals that a lot of + the software they provide is a link to the netlib repository. + + + + + Object Oriented Numerics + A site devoted to object oriented numerics. It has a Mailing list, + Extensive Links to freely available libraries (OO of course) + and freely available tools for object oriented scientific computing. + + + + + GNU Scientific Library + The GNU Scientific Library (GSL) is a collection of numerical routines + written from scratch in C. It provides an Applications Programming + Interface (API) for C programmers and also allows wrappers to be written + for very high level languages. It covers a wide range of numerical + computing topics, has a good manual, is widely portable and is distributed + under the GNU General Public License. + + + + + GiNaC + GiNaC is designed to allow the creation of software which need symbolic + manipulations embedded in them. It extends C++ by a set of algebraic + capabilities and is recursively named for GiNaC is not a Computer Algebra + system. It is distributed under the terms and conditions of the GNU + general public license (GPL). + + + + + + + Other topic specific numerical libraries + + + + FFTW + FFTW is a collection of fast C routines for computing the Discrete + Fourier Transform in one or more dimensions. It includes complex, + real, and parallel transforms, and can handle arbitrary array sizes + efficiently. This package includes both the double- and + single-precision FFTW uniprocessors and the threads libraries. + + + + + + LAPACK + LAPACK (Linear Algebra PACKage) is a standard library for numerical + linear algebra. LAPACK provides routines for solving systems of + simultaneous linear equations, least-squares solutions of linear + systems of equations, eigenvalue problems, and singular value + problems. LAPACK is coded in Fortran77 and is built with egcs. + It is well documented and widely used (and therefore widely tested). + + + + + + SuperLU + SuperLU is a general purpose library which performs an LU decomposition + for the direct solution of large, sparse, non-symmetric systems of linear + equations on high performance machines. Its written in C and is callable + from either C or Fortran. + + + + + + ARPACK + + ARPACK is a set of Fortran77 subroutines designed to solve large scale + eigenvalue problems. A Users Guide + is available. The above link also gives information about a + parallel version of ARPACK - PARPACK and a object oriented version + ARPACK++. + + + + + + + Computational Fluid Dynamics codes + This link contains a comprehensive listing of public domain, shareware + and freeware Computational Fluid Dynamics codes links with a description + of each CFD code. + + + + + + + + + Graphics and Visualization + + + + Gnuplot + Gnuplot is a command-line driven interactive function plotting + utility. It handles both curves (2 dimensions) and surfaces (3 + dimensions). Surfaces can be floating in the 3-d coordinate space, + or as a contour plot. For 2-d plots, there are also many plot + styles, including lines, points, lines with points, error bars, + and impulses. Graphs may be labeled with arbitrary labels and + arrows, axes labels, a title, date and time, and a key. It has + multiple plotting capabilities too. It allows saving the graphs in + various formats which can be included in word processors. + It can be used to generate publication quality plots. + + + + + + NCAR Graphics + A very popular graphics package which is very well documented and widely + used. It provides basic ingredients for creating complex plots as + functions / routines that can be called from Fortran and C. There is + a contributed programming interface to the NCAR Graphics package: + NCL (NCAR Command Language). The programming interfaces provide access + to complex graphics utilities like contouring, world map projections, + and velocity vectors. For the most part, the C interface is built on + top of the Fortran interface... It is distributed under the GNU public + license. Click here + for going to the documentation of all its various components. + + + + + OpenDX + A very good Open Source Data eXplorer. It can handle large amounts of data + and creates great visualizations. It was the tool I stumbled upon when + I wanted a free graphics routine to make 3-D plots and zoom-in, rotate, + and really eXplore the output Data from my codes. The downside is that + compiling from source is really challenging and getting started is a + difficult. However it has excellent documentation distributed with it + and once I started off it was the best tool I have ever used. + + + + + Gri: + It is a language for scientific graphics programming. The claim that Gri + is similar to LaTeX in the sense that both provide extensive power as a + reward for tolerating a learning curve seems exciting and I for one want + to check this out!! Check out the following + article + in The Linux Journal. Go to the gri home page if you are now impressed by + it and check out download info and manuals. + + + + + MayaVi: + A scientific data visualizer written in Python. It is distributed under the + BSD + license. The screenshots look promising. Check out the above link + for more details. + + + + + PGPLOT: + PGPLOT is a Fortran 77 or C callable subroutine package for drawing + scientific 2D and Simple 3D plots. One can call these routines during + runtime and redirect the output to a variety of devices at run time. + It is well documented and the full documentation is available at the + above site. It is Free for Non-Commercial Use. A user manual is available + online at PGPLOT + Users Manual + + + + + PLplot: + This is a library of scientific plotting functions that can be called from + C, C++, FORTRAN, TCL, PYTHON. PLplot features as described in the above link + are, "It can be used to create standard x-y plots, semilog plots, + log-log plots, contour plots, 3D plots, mesh plots, bar charts and pie charts. + Multiple graphs (of the same or different sizes) may be placed on a single + page with multiple lines in each graph. There are almost 2000 characters + in the extended character set. This includes four different fonts, the + Greek alphabet and a host of mathematical, musical, and other symbols. + A variety of output devices are supported and new devices can be easily + added by writing a small number of device dependent routines". To download + click + + here . + + + + + Grace + Grace is a WYSIWYG 2D plotting tool for the X Window System and Motif. + Grace runs on practically any version of Unix. Grace is a descendant of + ACE/gr, also known as Xmgr. It is lisenced under the GNU public license. + This link also has a tutorial and download information. + + + + + SciGraphica + + SciGraphica is a application for data analysis and technical graphics. + It fully supplies plotting features for 2D, 3D and polar charts. The + aim is to obtain a fully-featured, cross-platform, user-friendly, + self-growing scientific application. It is free and open-source, + released under the GPL license. + + + + + + Plotutils: + The GNU plotutils package contains software for both programmers and + technical users. Its centerpiece is libplot.a powerful C/C++ function + library for exporting 2-D vector graphics in many file formats, both + vector and raster. It can also do vector graphics animations. + Besides libplot, the package contains command-line programs for + plotting scientific data. Many of them use libplot to export graphics. + + + + + DISLIN + DISLIN is a high-level plotting library for displaying data as curves, + polar plots, bar graphs, pie charts, 3D-color plots, surfaces, contours + and maps. + + + + + ImLib3D + ImLib3D is an open source C++ library for 3D (volumetric) image processing. + It contains most basic image processing algorithms, and some more + sophisticated ones. It comes with an optional viewer that features + multi-planar views, animations, vector field views and 3D (OpenGL) + multi-planar. + + + + + Ptplot: + Ptplot is a 2D data plotter and histogram tool implemented in Java. + Ptplot can be used as a standalone applet or application, or it can be + embedded in your own applet or application. + + + + + + Programming systems for GNU/Linux + + This section deals with links to tutorials and documents for installing + Linux on a PC, getting started with Linux, and then going a step further + -- to optimize your PC for processing power, using multiple processors + (Symmetric Muliti Processing - SMP); making a cheap, upgradeable + Supercomputing Linux cluster and finally links to software to do parallel + programming on Linux. + + + The GNU/Linux Workstation + + As with most documentation related to GNU/Linux, + the Linux Documentation project's + home page is a priceless source. You might first want to read + + The Linux Installation HOWTO. For those who want to install + Linux along with Windows might want to browse through + + The Linux + Windows HOWTO. When installing Linux make sure + that you choose to install all documentation. After installing Linux, + a good, comprehensive document to getting started with using Linux is + The Rute Users Tutorial and + Exposition which is a beginners guide to Linux and UNIX like + systems. I'd like to give a less intimidating (size-wise) link to a + small beginners guide, but U will find this useful after taking the + plunge. You might also want to go through The Linux System + Administrator's Guide and to check out + The Linux Administration Made Easy (LAME) guide + It attempts to describe day-to-day administration and maintenance + issues commonly faced by Linux system administrators. + + + + Parallel Processing and Symmetric Multiprocessing: + Supercomputing + + It is possible to get large volume number crunching without + spending millions of rupees on a supercomputer. You only need + to link together (by some high speed network) the requisite + number of CPUs, with GNU/LINUX as the underlying OS. Add + some freely available message passing software and a effective + parallel processing number crunching machine is made. Such + clusters are called "Beowulf clusters". The other advantages + of such a cluster other than building costs is, up-gradation + costs are minimal. The two best resources for Linux cluster + builders are + + + The Beowulf + Project home page and + + + The Extreme + Linux Project + + + + These sites are upgraded frequently with useful information + for cluster builders. + + + Parallel computing document links + + You will also want to read this excellent article on + Linux + Clustering Software (and the large variety of links + it provides) by Joe Greenseid. I hope to go through the links + and include them subsequently in this HOWTO. + + + + Other free document links for parallel processing are: + + + + + + The Beowulf Howto : + This document introduces the Beowulf Supercomputer architecture + and provides background information on parallel programming, + including links to other more specific documents, and web pages. + But, before that for an understanding of parallel processing and + Symmetric multiprocessing on Linux, check out the following: + + + + + + The Parallel Processing on Linux HOWTO + + + + + + The Symmetric Multiprocessing HOWTO + + + + + + + Parallel processing software for Linux + + Now after reading the above documents, you have an idea of parallel + processing. Parallel program libraries are the core of parallel processing + on a Linux cluster. There are various free implementations of parallel + processing libraries. Since parallel processing is all about performance, + these libraries have some very nice functional tools to analyze your parallel + program performance. Given below is a set of links to these parallel + program libraries and tools. + + + + + + + + Message Passing Interface: + MPI is a standard specification of message passing libraries. The above + document gives a lot of links to documents on the standard, etc.. A MPI + implementation for Linux + mpich is also + available at that site. There are a lot of documents for + Learning to use MPI + . + + + + + + Local Area Multicomputer + - LAM: + LAM (Local Area Multicomputer) is an MPI programming environment and + development system for heterogeneous computers on a network. + With LAM, a dedicated cluster or an existing network computing + infrastructure can act as one parallel computer solving one problem. + LAM features extensive debugging support in the application development + cycle and peak performance for production applications. LAM features a + full implementation of the MPI communication standard. + You can download the sources (tar-zipped, rpm) or binaries from + here + A host of MPI tutorial links and also a `getting started with LAM' + tutorial is available + here + + + + + + + Parallel Virtual Machine : + As the PVM home page describes, it is a software package that permits + a heterogeneous collection of Unix and/or NT computers hooked together + by a network to be used as a single large parallel computer. Thus large + computational problems can be solved more cost effectively by using the + aggregate power and memory of many computers. The software is very + portable. The source, which is available free thru netlib, has been + compiled on everything from laptops to CRAYs. + + + + + + Ganglia: + Ganglia is an open source cluster monitoring and execution environment + developed at the University of California, Berkeley Computer Science + Division. As the above link describes it, "Ganglia is as simple to + install and use on a 16-node cluster as it is to use on a 512-node + cluster as has been proven by its use on multiple 500+ node clusters". + It not only can link nodes in a cluster, but also link clusters to other + clusters. + + + + + + + + + + Word-Processing and Poster presenting tools on Linux + + + Those of you who do not use + LaTeX and find it challenging and want a WYSIWYG word processor, + keep your ears tuned to OpenOffice + + which has released version 1.2 of its openoffice software. Its tools + may compare well with the best in the market. + + + + Word Processing Tools + + + + + Latex: + LaTeX is a high-quality typesetting system, with features designed for + the production of technical and scientific documentation. LaTeX is the + de facto standard for the communication and publication of scientific + documents. + + David R.Wilkin's primer "Getting Started with LaTeX" + is a good tutorial to getting started with LaTeX. + For those who have to live with a WYSIWYG documenting tool, check + LyX. This is a front-end for + latex. It isn't as powerful as latex proper, but helps with a good WYSIWIG. + + + + + + Lout: + A document formatting system similar to latex. Good features, documentation + and history. Light weight and outputs postscript. Thanks to Emiliano Gavilan + for this link. + + + + + + Abiword: + As the AbiWord home page says, + "AbiWord is suitable for typing papers, letters, reports, memos, and so forth". + It has won many awards and seems to be the best open source WYSIWYG word + processor. Check out the above link to know more about it and download it. + + + + + + kword: + As the kword home page says, + "KWord is a FrameMaker-like word-processing and desktop publishing application. + KWord is capable of creating demanding and professional looking documents. + Whether you are a corporate or home user, production artist or student, KWord + will prove a valuable and easy to use tool for all your word processing and + layout needs". Check out the above link to know more about it and download it. + (you might want to know more about the whole + koffice suite). + + + + + + + Poster presentation tools + + + + + KPresenter: + KPresenter is the presentation tool of the KOffice suite of office + utilities. It allows screen presentations with all the trappings one + is used to seeing in costly presentation tools. It also allows honest, + real scientific presentations where one does not have to impress the + audience with non subject specific stuff :-). The best thing about it + is the possibility of saving the presentation as a html file. It makes + portable network graphics files with each presentation slide. With a + smattering of knowledge of html files one could put in a animated image + as a image link thereby allowing one to show movies too when necessary. + + + + + + Xfig : + Though the man page claims that it is a facility for the + interactive generation of figures ...., It in fact much more than that. + Other than generating figures for elucidating what you want to say in + a poster, it helps you import and export figures in a variety of formats, + write text in various fonts and sizes, generate Greek symbols and color + text, Save as latex picture file or any other format supported by your + word processor for inclusion in your publications, generate GIFs of each + page of the poster to put on your web site, and finally it generates + *.fig files which are small in size. The only thing on my wish list for + xfig is the capability to edit the imported pictures which are not in + *.fig format. Therefore for a computer screen projected poster + presentation you need a frames capable browser with contents in one frame + and the xfig generated posters (exported as *.png or *.jpg from Xfig) on + the other. + + + + + + + + Free Database Management Systems for Linux + + Scientific computing has two parallel data needs, one the physical values + of the data itself, and the other is Database systems to manage the data. + In this document links are provided only to database resources on the net + and free Database systems. I personally do not use databases to manipulate + the data generated by my codes. gawk, sed, and other basic Unix commands + like grep, head and tail seem sufficient to manipulate any data I generate. + I thought I should include this section for the large data generators. + Hopefully a person with experience in databases will make this section better. + + + + + + + + Free database list + + + + + + + ACM SIGMOD: + Index of publicly available database software. + + + + + + MySQL: + A relational Database management system. + + + + + + PostgreSQL + As the link above describes it ...PostgreSQL is + a sophisticated Object-Relational DBMS, supporting almost all SQL + constructs, including subselects, transactions, and user-defined types + + + + + + + + Linux in the laboratory + + Again this is a section where I have zero experience and hope someone will + with experience will contribute towards making this document better. However, + I provide below links suggested by Sambaran Pahari and Deepak Gupta. These + links seem to be very good from my inexperienced viewpoint. + + + + + The Linux Lab Project + A site for "Linux Lab Project."..everything to do with laboratory process, + process control, automation and data acquisition on Linux. + As the above link says, "The Linux lab project is intended to help people with + development of data collection and process control software for LINUX. + It is planned to provide a standardized development environment for a wide variety + of applications from hardware support to application development". + + + + + + Linux Parallel port drivers: + The above link says, "If you have a parallel port device and would + like to know if there is a Linux driver available for it --then this is + the place to look". Sounds like a confident claim. + + + + + +