diff --git a/LDP/howto/linuxdoc/Clusters-HOWTO.sgml b/LDP/howto/linuxdoc/Clusters-HOWTO.sgml new file mode 100644 index 00000000..a3f3826b --- /dev/null +++ b/LDP/howto/linuxdoc/Clusters-HOWTO.sgml @@ -0,0 +1,321 @@ + + +
+ +Linux Cluster HOWTO + +Ram Samudrala (me@ram.org) + v0.2, June 10, 2001 + + +How to set up high-performance Linux computing clusters. + + + + + + + Introduction + +

This document describes how I set up my Linux computing clusters +for high-performance computing which I need for .

+ +

Use the information below at your own risk. I disclaim all +responsibility for anything you may do after reading this HOWTO. The +latest version of this HOWTO will always be available at .

+ +

Unlike other documentation that talks about setting up clusters in +a general way, this is a specific description of how our lab is setup +and includes not only details the compute aspects, but also the +desktop, laptop, and public server aspects. This is done mainly for +local use, but I figure I might as well put it up on the web and +perhaps someone else will find it useful. The main use as it stands +is that it's a report on what kind of hardware works well with Linux +and what kind of hardware doesn't.

+ + + + + + + + Hardware + +

This section covers the hardware choices I've made. Unless noted, +assume that everything works really well.

+ + + + Node hardware + +

32 machines have the following setup each: + + + 2 Pentium III 1 GHz Intel CPUs + Supermicro 370 DLE Dual PIII-FCPGA motherboard + 2 256 MB 168-pin PC133 Registered ECC Micron RAM + 1 20 GB Maxtor ATA/66 5400 RPM HD + 1 40 GB Maxtor UDMA/100 7200 RPM HD + Asus CD-S500 50x CDROM + 1.4 MB floppy drive + ATI Expert 98 8 MB PCI video card + Mid-tower case + +

+ + + + + + Server hardware + +

1 external server with the following setup: + + + 2 Pentium III 1 GHz Intel CPUs + Supermicro 370 DLE Dual PIII-FCPGA motherboard + 2 256 MB 168-pin PC133 Registered ECC Micron RAM + 1 20 GB Maxtor ATA/66 5400 RPM HD + 2 40 GB Maxtor UDMA/100 7200 RPM HD + Asus CD-S500 50x CDROM + 1.4 MB floppy drive + ATI Expert 98 8 MB PCI video card + Full-tower case + + + + + + + Desktop hardware + +

4 desktops with the following setup: + + + 2 Pentium III 1 GHz Intel CPUs + Supermicro 370 DE6 Dual PIII-FCPGA motherboard + 4 256 MB 168-pin PC133 Registered ECC Micron RAM + 3 40 GB Maxtor UDMA/100 7200 RPM HD + Ricoh 32x12x10 CDRW/DVD Combo EIDE + 1.4 MB floppy drive + Asus V7700 64mb GeForce2-GTS AGP video card + Creative SB Live Platinum 5.1 sound card + Microsoft Natural Keyboard + Microsoft Intellimouse Explorer + Full-tower case + +

+ +

2 desktops with the following setup: + + + 2 Pentium III 1 GHz Intel CPUs + Supermicro 370 DLE Dual PIII-FCPGA motherboard + 4 256 MB 168-pin PC133 Registered ECC Micron RAM + 3 40 GB Maxtor UDMA/100 7200 RPM HD + Mitsumi 8x/4x/32x CDRW + 1.4 MB floppy drive + Jaton Nvidia TNT2 32mb PCI + Creative SB LIVE Value PCI + Microsoft Natural Keyboard + Microsoft Intellimouse Explorer + Full-tower case + +

+ +

2 desktops with the following setup: + + + 2 Pentium III 1 GHz Intel CPUs + Supermicro 370 DLE Dual PIII-FCPGA motherboard + 4 256 MB 168-pin PC133 Registered ECC Micron RAM + 3 40 GB Maxtor UDMA/100 7200 RPM HD + Asus CD-S500 50x CDROM + 1.4 MB floppy drive + Jaton Nvidia TNT2 32mb PCI + Creative SB LIVE Value PCI + Microsoft Natural Keyboard + Microsoft Intellimouse Explorer + Full-tower case + +

+ +

Backup: + + + 2 Sony 20/40 GB DSS4 SE LVD DAT + +

+ +

Monitors: + + + 4 21" Sony CPD-G500 .24mm monitor + 2 18" Viewsonic VP-181 TFT-LCD monitor + +

+ + + + + + Putting-it-all-together hardware + +

We use KVM switches with a cheap monitor to connect up and "look" +at all the machines: + + + 15" .28dp XLN CTL Monitor + 3 Belkin Omniview 16-Port Pro Switches + 40 KVM cables + +

+ +

Networking is important. + + + 1 Cisco Catalyst 3448 XL Enterprise Edition network switch. + +

+ + + + + + Costs + +

Our vendor is Hard Drives Northwest (). For each +compute node in our cluster (containing two processors), we paid about +$1500, including taxes. Generally, our goal is to keep each node to +below $2000.00 (which is what our desktop machines cost).

+ + + + + + + + + + Software + + + + Linux, of course! + +

Specfically we use 2.2.17-14 kernel based on the KRUD 7.0 +distribution. We use our own software for parallising applications but +have experimented with PVM and MPI. In my view, the overhead for these +pre-packaged programs is too high.

+ + + + Costs + +

Linux is freely copiable.

+ + + + + + + + + + Configuration + + + + Disk configuration + +

This section describes disk partitioning strategies.

+ +

+ +desktops (without windows): + +hda1 - swap (2 * RAM) +hda2 - / (4 GB) +hda3 - /home (remaining disk space) +hdb1 - /maxa (total disk) +hdd1 - /maxb (total disk) + +desktops (with windows): + +hda1 - /win (total disk) +hdb1 - swap (2 * RAM) +hdb2 - / (4 GB) +hdb3 - /home (remaining disk space) +hdd1 - /maxa (total disk) + +laptops (single disk): + +hda1 - /win (half the total disk size) +hda2 - swap (2 * RAM) +hda3 - / (4 GB) +hda4 - /home (remaining disk space) + +farm machines: + +hda1 - swap (2 * RAM) +hda2 - / (remaining disk space) +hdb1 - /maxa (total disk) + +

+ + + + + + Package configuration + +

Install a minimal set of packages for the farm. Users are allowed +to configure desktops as they wish.

+ + + + + + + + Acknowledgements + +

The following people have been helpful in getting this HOWTO +done: + + + Michael Levitt () + +

+ + + + + + Bibliography + + + +
+