old-www/HOWTO/openMosix-HOWTO/x332.html

828 lines
15 KiB
HTML

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML
><HEAD
><TITLE
>openMosix General Instructions</TITLE
><META
NAME="GENERATOR"
CONTENT="Modular DocBook HTML Stylesheet Version 1.7"><LINK
REL="HOME"
TITLE="The openMosix HOWTO"
HREF="index.html"><LINK
REL="UP"
TITLE="Distribution specific installations"
HREF="distributions.html"><LINK
REL="PREVIOUS"
TITLE="Getting openMosix"
HREF="x326.html"><LINK
REL="NEXT"
TITLE="Red Hat and openMosix"
HREF="x424.html"></HEAD
><BODY
CLASS="SECT1"
BGCOLOR="#FFFFFF"
TEXT="#000000"
LINK="#0000FF"
VLINK="#840084"
ALINK="#0000FF"
><DIV
CLASS="NAVHEADER"
><TABLE
SUMMARY="Header navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TH
COLSPAN="3"
ALIGN="center"
>The openMosix HOWTO: </TH
></TR
><TR
><TD
WIDTH="10%"
ALIGN="left"
VALIGN="bottom"
><A
HREF="x326.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="80%"
ALIGN="center"
VALIGN="bottom"
>Chapter 4. Distribution specific installations</TD
><TD
WIDTH="10%"
ALIGN="right"
VALIGN="bottom"
><A
HREF="x424.html"
ACCESSKEY="N"
>Next</A
></TD
></TR
></TABLE
><HR
ALIGN="LEFT"
WIDTH="100%"></DIV
><DIV
CLASS="SECT1"
><H1
CLASS="SECT1"
><A
NAME="AEN332"
></A
>4.4. openMosix General Instructions</H1
><DIV
CLASS="SECT2"
><H2
CLASS="SECT2"
><A
NAME="AEN334"
></A
>4.4.1. Kernel Compilation</H2
><P
>&#13;Always use pure vanilla kernel-sources from <A
HREF="http://www.kernel.org/"
TARGET="_top"
><I
CLASS="CITETITLE"
>http://www.kernel.org/</I
></A
> to compile an
openMosix kernel! Please be kind enough to download the kernel using a
mirror near to you and always try and download patches to the latest
kernel sources you do have instead of downloading the whole
thing. This is going to be much appreciated by the Linux community and
will greatly increase your geeky Karma ;-)
Be sure to use the right openMosix patch depending on the
kernel-version. At the moment I write this, the latest 2.4 kernel is
2.4.20 so you should download the openMosix-2.4.20-x.gz patch, where
the "x" stands for the patch revision (ie: the greater the revision
number, the most recent it is).
Do not use the kernel that comes with any Linux-distribution: it won't
work. These kernel sources get heavily patched by the
distribution-makers so, applying the openMosix patch to such a kernel
is going to fail for sure! Been there, done that: trust me ;-)</P
><P
>Download the actual version of the openMosix patch and move it in your
kernel-source directory (e.g. /usr/src/linux-2.4.20). If your
kernel-source directory is other than
"/usr/src/linux-[version_number]" at least the creation of a symbolic
link to "/usr/src/linux-[version_number]" is required.
Supposing you're the root user and you've downloaded the gzipped patch
file in your home directory, apply the patch using (guess what?) the
patch utility:
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>mv /root/openMosix-2.4.20-2.gz /usr/src/linux-2.4.20
cd /usr/src/linux-2.4.20
zcat openMosix-2.4.20-2.gz | patch -Np1</PRE
></FONT
></TD
></TR
></TABLE
>
In the rare case you don't have "zcat" on your system, do:
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>mv /root/openMosix-2.4.20-2.gz /usr/src/linux-2.4.20
cd /usr/src/linux-2.4.20
gunzip openMosix-2.4.20-2.gz
cat openMosix-2.4.20-2 | patch -Np1</PRE
></FONT
></TD
></TR
></TABLE
>
If the even more weird case you don't have a "cat" on your system (!),
do:
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>mv /root/openMosix-2.4.20-2.gz /usr/src/linux-2.4.20
cd /usr/src/linux-2.4.20
gunzip openMosix-2.4.20-2.gz
patch -Np1 &#60; openMosix-2.4.20-2</PRE
></FONT
></TD
></TR
></TABLE
>
The "patch" command should now display a list of patched files from
the kernel-sources.
If you feel adventurous enough, enable the openMosix related options
in the kernel-configuration file, e.g.
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>...
CONFIG_MOSIX=y
# CONFIG_MOSIX_TOPOLOGY is not set
CONFIG_MOSIX_UDB=y
# CONFIG_MOSIX_DEBUG is not set
# CONFIG_MOSIX_CHEAT_MIGSELF is not set
CONFIG_MOSIX_WEEEEEEEEE=y
CONFIG_MOSIX_DIAG=y
CONFIG_MOSIX_SECUREPORTS=y
CONFIG_MOSIX_DISCLOSURE=3
CONFIG_QKERNEL_EXT=y
CONFIG_MOSIX_DFSA=y
CONFIG_MOSIX_FS=y
CONFIG_MOSIX_PIPE_EXCEPTIONS=y
CONFIG_QOS_JID=y
...</PRE
></FONT
></TD
></TR
></TABLE
>
However, it's going to be pretty much easier if you configure the
above options using one of the Linux-kernel configuration tools:
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>make config | menuconfig | xconfig</PRE
></FONT
></TD
></TR
></TABLE
>
The above means you have to choose one of "config", "menuconfig", and
"xconfig". It's a matter of taste. By the way, "config" is going to
work on any system; "menuconfig" needs the curses libraries installed
while "xconfig" needs an installed X-window environment plus the
TCL/TK libraries and interpreters.</P
><P
>Now compile it with:
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>make dep bzImage modules modules_install</PRE
></FONT
></TD
></TR
></TABLE
>
After compilation install the new kernel with the openMosix options
within you boot-loader; e.g. insert an entry for the new kernel in
/etc/lilo.conf and run lilo after that.</P
><P
>Reboot and your openMosix-cluster-node is up!</P
></DIV
><DIV
CLASS="SECT2"
><H2
CLASS="SECT2"
><A
NAME="AEN348"
></A
>4.4.2. Syntax of the /etc/openmosix.map file</H2
><P
>Before starting openMosix, there has to be an /etc/openmosix.map
configuration file which must be the same on each node. </P
><P
>The standard is now /etc/openmosix.map, /etc/mosix.map and /etc/hpc.map are old
standards, but the CVS-version of the
tools is backwards compatible and looks for /etc/openmosix.map,
/etc/mosix.map and /etc/hpc.map (in that order). </P
><P
>The
openmosix.map file contains three space separated fields:
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>openMosix-Node_ID IP-Address(or hostname) Range-size</PRE
></FONT
></TD
></TR
></TABLE
>
An example openmosix.map file could look like this:
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>1 node1 1
2 node2 1
3 node3 1
4 node4 1</PRE
></FONT
></TD
></TR
></TABLE
>
or
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>1 192.168.1.1 1
2 192.168.1.2 1
3 192.168.1.3 1
4 192.168.1.4 1</PRE
></FONT
></TD
></TR
></TABLE
>
or with the help of the range-size both of the above examples equal to:
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>1 192.168.1.1 4</PRE
></FONT
></TD
></TR
></TABLE
>
openMosix "counts-up" the last byte of the ip-address of the node
according to its openMosix-Node_ID. Of course, if you use a
range-size greater than 1 you have to use ip-addresses instead of
hostnames.</P
><P
>If a node has more than one network-interface it can be configured
with the ALIAS option in the range-size field (which equals to setting
the range-size to 0) e.g.
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>1 192.168.1.1 1
2 192.168.1.2 1
3 192.168.1.3 1
4 192.168.1.4 1
4 192.168.10.10 ALIAS</PRE
></FONT
></TD
></TR
></TABLE
>
Here the node with the openMosix-Node_ID 4 has two network-interfaces
(192.168.1.4 + 192.168.10.10) which are both visible to openMosix.</P
><P
><EM
>Always be sure to run the same openMosix version AND configuration on each
of your Cluster's nodes!</EM
></P
><P
>Start openMosix with the "setpe" utility on each node :
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>setpe -w -f /etc/openmosix.map</PRE
></FONT
></TD
></TR
></TABLE
>
Execute this command (which will be described later on in this HOWTO)
on every node in your openMosix cluster.</P
><P
>Alternatively, you can grab the "openmosix" script which can be found
in the scripts directory of the userspace-tools, copy it to the
/etc/init.d directory, chmod 0755 it, then use the following commands
as root:
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>/etc/init.d/openmosix stop
/etc/init.d/openmosix start
/etc/init.d/openmosix restart</PRE
></FONT
></TD
></TR
></TABLE
></P
><P
>Installation is finished now: the cluster is up and running :)</P
></DIV
><DIV
CLASS="SECT2"
><H2
CLASS="SECT2"
><A
NAME="AEN366"
></A
>4.4.3. oMFS</H2
><P
>First of all, the CONFIG_MOSIX_FS option in the kernel configuration
has to be enabled. If the current kernel was compiled without this
option, then recompilation with this option enabled is required.</P
><P
> Also
the UIDs (User IDs) and GIDs (Group IDs) on each of the clusters'
nodes file-systems must be the same. You might want to accomplish this using
openldap. The CONFIG_MOSIX_DFSA option in
the kernel is optional but of course required if DFSA should be used.
To mount oMFS on the cluster there has to be an additional fstab-entry
on each node's /etc/fstab.</P
><P
>in order to have DFSA enabled:
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>mfs_mnt /mfs mfs dfsa=1 0 0</PRE
></FONT
></TD
></TR
></TABLE
>
in order to have DFSA disabled:
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>mfs_mnt /mfs mfs dfsa=0 0 0</PRE
></FONT
></TD
></TR
></TABLE
>
the syntax of this fstab-entry is:
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>[device_name] [mount_point] mfs defaults 0 0</PRE
></FONT
></TD
></TR
></TABLE
>
After mounting the /mfs mount-point on each node, each node's
file-system is going to be accessible through the
/mfs/[openMosix-Node_ID]/ directories.</P
><P
>With the help of some symbolic links all cluster-nodes can access the same
data e.g. /work on node1
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>on node2 : ln -s /mfs/1/work /work
on node3 : ln -s /mfs/1/work /work
on node3 : ln -s /mfs/1/work /work
...</PRE
></FONT
></TD
></TR
></TABLE
>
Now every node can read+write from and to /work !</P
><P
>&#13;The following special files are excluded from the oMFS:
<P
></P
><UL
><LI
STYLE="list-style-type: opencircle"
><P
>the /proc directory</P
></LI
><LI
STYLE="list-style-type: opencircle"
><P
>special files which are not regular-files,
directories or symbolic links (e.g. /dev/hda1) </P
></LI
></UL
></P
><P
>Creating links like:
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>ln -s /mfs/1/mfs/1/usr </PRE
></FONT
></TD
></TR
></TABLE
>
or
<TABLE
BORDER="0"
BGCOLOR="#E0E0E0"
WIDTH="100%"
><TR
><TD
><FONT
COLOR="#000000"
><PRE
CLASS="PROGRAMLISTING"
>ln -s /mfs/1/mfs/3/usr</PRE
></FONT
></TD
></TR
></TABLE
>
is invalid.</P
><P
>The following system calls are supported without sending the migrated
process (which executes this call on its home (remote) node) going
back to its home node:</P
><P
>read, readv, write, writev, readahead, lseek, llseek, open, creat,
close, dup, dup2, fcntl/fcntl64, getdents, getdents64, old_readdir,
fsync, fdatasync, chdir, fchdir, getcwd, stat, stat64, newstat, lstat,
lstat64, newlstat, fstat, fstat64, newfstat, access, truncate,
truncate64, ftruncate, ftruncate64, chmod, chown, chown16, lchown,
lchown16, fchmod, fchown, fchown16, utime, utimes, symlink, readlink,
mkdir, rmdir, link, unlink, rename</P
><P
>Here are situations when system calls on DFSA mounted file-systems may not
work:
<P
></P
><UL
><LI
STYLE="list-style-type: opencircle"
><P
>different mfs/dfsa configuration on the
cluster-nodes</P
></LI
><LI
STYLE="list-style-type: opencircle"
><P
>dup2 if the second file-pointer is
non-DFSA</P
></LI
><LI
STYLE="list-style-type: opencircle"
><P
>chdir/fchdir if the parent dir is
non-DFSA</P
></LI
><LI
STYLE="list-style-type: opencircle"
><P
>pathnames that leave the
DFSA-filesystem</P
></LI
><LI
STYLE="list-style-type: opencircle"
><P
>when the process which executes the system-call is
being traced</P
></LI
><LI
STYLE="list-style-type: opencircle"
><P
>if there are pending requests for the process which
executes the system-call</P
></LI
></UL
></P
><P
>Next to the /mfs/1/ /mfs/2/ and so on files you will find some other
directories as well.</P
><DIV
CLASS="TABLE"
><A
NAME="AEN402"
></A
><P
><B
>Table 4-1. Other Directories</B
></P
><TABLE
BORDER="1"
CLASS="CALSTABLE"
><TBODY
><TR
><TD
ALIGN="LEFT"
VALIGN="TOP"
>/mfs/here</TD
><TD
ALIGN="LEFT"
VALIGN="TOP"
>The current node where your process runs</TD
></TR
><TR
><TD
ALIGN="LEFT"
VALIGN="TOP"
>/mfs/home</TD
><TD
ALIGN="LEFT"
VALIGN="TOP"
>Your home node</TD
></TR
><TR
><TD
ALIGN="LEFT"
VALIGN="TOP"
>/mfs/magic</TD
><TD
ALIGN="LEFT"
VALIGN="TOP"
>The current node when used by the "creat" system call (or
an "open" with the "O_CREAT" option) - otherwise, the last node on
which an oMFS magical file was successfully created (this is very
useful for creating temporary-files, then immediately unlinking
them)
</TD
></TR
><TR
><TD
ALIGN="LEFT"
VALIGN="TOP"
>/mfs/lastexec</TD
><TD
ALIGN="LEFT"
VALIGN="TOP"
>The node on which the process last issued a successful
"execve" system-call.
</TD
></TR
><TR
><TD
ALIGN="LEFT"
VALIGN="TOP"
>/mfs/selected</TD
><TD
ALIGN="LEFT"
VALIGN="TOP"
>The node you selected by either your process itself or one
of its ancestor's (before forking this process), writing a number
into "/proc/self/selected".
</TD
></TR
></TBODY
></TABLE
></DIV
><P
>&#13;Note that these magic files are all ``per process''. That is their
content is dependent upon which process opens them.</P
><P
>A last not about openMFS is that there are versions around that return
faultive results when you run <SPAN
CLASS="QUOTE"
>"df"</SPAN
> on those filesystems.
Don't be surpised if you suddenlty have about 1.3 TB available on those
systems.</P
></DIV
></DIV
><DIV
CLASS="NAVFOOTER"
><HR
ALIGN="LEFT"
WIDTH="100%"><TABLE
SUMMARY="Footer navigation table"
WIDTH="100%"
BORDER="0"
CELLPADDING="0"
CELLSPACING="0"
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
><A
HREF="x326.html"
ACCESSKEY="P"
>Prev</A
></TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
><A
HREF="index.html"
ACCESSKEY="H"
>Home</A
></TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
><A
HREF="x424.html"
ACCESSKEY="N"
>Next</A
></TD
></TR
><TR
><TD
WIDTH="33%"
ALIGN="left"
VALIGN="top"
>Getting openMosix</TD
><TD
WIDTH="34%"
ALIGN="center"
VALIGN="top"
><A
HREF="distributions.html"
ACCESSKEY="U"
>Up</A
></TD
><TD
WIDTH="33%"
ALIGN="right"
VALIGN="top"
>Red Hat and openMosix</TD
></TR
></TABLE
></DIV
></BODY
></HTML
>