540 lines
13 KiB
HTML
540 lines
13 KiB
HTML
<HTML
|
|
><HEAD
|
|
><TITLE
|
|
>Client-to-Server Farming w/ Round Robin DNS</TITLE
|
|
><META
|
|
NAME="GENERATOR"
|
|
CONTENT="Modular DocBook HTML Stylesheet Version 1.76b+
|
|
"><LINK
|
|
REL="HOME"
|
|
TITLE="Jabber Server Farming How-To"
|
|
HREF="index.html"><LINK
|
|
REL="UP"
|
|
TITLE="Implementing the Farm"
|
|
HREF="implementation.html"><LINK
|
|
REL="PREVIOUS"
|
|
TITLE="Implementing the Farm"
|
|
HREF="implementation.html"><LINK
|
|
REL="NEXT"
|
|
TITLE="Connection Redirection"
|
|
HREF="redirection.html"></HEAD
|
|
><BODY
|
|
CLASS="section"
|
|
BGCOLOR="#FFFFFF"
|
|
TEXT="#000000"
|
|
LINK="#0000FF"
|
|
VLINK="#840084"
|
|
ALINK="#0000FF"
|
|
><DIV
|
|
CLASS="NAVHEADER"
|
|
><TABLE
|
|
SUMMARY="Header navigation table"
|
|
WIDTH="100%"
|
|
BORDER="0"
|
|
CELLPADDING="0"
|
|
CELLSPACING="0"
|
|
><TR
|
|
><TH
|
|
COLSPAN="3"
|
|
ALIGN="center"
|
|
>Jabber Server Farming How-To</TH
|
|
></TR
|
|
><TR
|
|
><TD
|
|
WIDTH="10%"
|
|
ALIGN="left"
|
|
VALIGN="bottom"
|
|
><A
|
|
HREF="implementation.html"
|
|
ACCESSKEY="P"
|
|
>Prev</A
|
|
></TD
|
|
><TD
|
|
WIDTH="80%"
|
|
ALIGN="center"
|
|
VALIGN="bottom"
|
|
>Chapter 3. Implementing the Farm</TD
|
|
><TD
|
|
WIDTH="10%"
|
|
ALIGN="right"
|
|
VALIGN="bottom"
|
|
><A
|
|
HREF="redirection.html"
|
|
ACCESSKEY="N"
|
|
>Next</A
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><HR
|
|
ALIGN="LEFT"
|
|
WIDTH="100%"></DIV
|
|
><DIV
|
|
CLASS="section"
|
|
><H1
|
|
CLASS="section"
|
|
><A
|
|
NAME="c2sfarm">3.1. Client-to-Server Farming w/ Round Robin DNS</H1
|
|
><DIV
|
|
CLASS="section"
|
|
><H2
|
|
CLASS="section"
|
|
><A
|
|
NAME="c2sfarmdream">3.1.1. The Dream</H2
|
|
><P
|
|
> The first thing that we can do is to split out
|
|
the c2s from the server and make it a seperate
|
|
process/component. Once that is done, we can
|
|
replicate the c2s processes over multiple
|
|
machines. Each one would connect back to the
|
|
main server running the JSM and everything
|
|
should work just fine.
|
|
</P
|
|
><DIV
|
|
CLASS="figure"
|
|
><A
|
|
NAME="AEN64"><P
|
|
><B
|
|
>Figure 3-1. c2s Farming Diagram</B
|
|
></P
|
|
><DIV
|
|
CLASS="mediaobject"
|
|
><P
|
|
><IMG
|
|
SRC="c2s-round-robin.jpeg"></P
|
|
></DIV
|
|
></DIV
|
|
><P
|
|
>Pros:</P
|
|
><P
|
|
></P
|
|
><UL
|
|
><LI
|
|
><P
|
|
>Easy to setup right now.</P
|
|
></LI
|
|
></UL
|
|
><P
|
|
>Cons:</P
|
|
><P
|
|
></P
|
|
><UL
|
|
><LI
|
|
><P
|
|
> Round Robin will not provide true load
|
|
balancing since there is no mechanism in
|
|
place to check how many connections a
|
|
server has when it forwards a new connection
|
|
to it the next time it comes up in the
|
|
Round Robin.
|
|
</P
|
|
></LI
|
|
></UL
|
|
></DIV
|
|
><DIV
|
|
CLASS="section"
|
|
><H2
|
|
CLASS="section"
|
|
><A
|
|
NAME="c2sfarmreality">3.1.2. The Reality</H2
|
|
><P
|
|
> This is doable right now. The 1.5 version of
|
|
jabberd has sought to break all of the pieces
|
|
out into seperate processes. This was not
|
|
done for farming specifically, but we will not
|
|
complain since the jadc2s component can handle
|
|
upwards of 10k users. (The 1.4.x series c2s
|
|
could only handle ~1024).
|
|
</P
|
|
><P
|
|
> Currently there is a forked version of jadc2s
|
|
that works with the 1.4.2 server. It is located
|
|
in the jabberd14 CVS repository on JabberStudio.
|
|
The following example is running with two jadc2s
|
|
boxes, and one central jabberd box. To set this
|
|
up do the following:
|
|
</P
|
|
><P
|
|
></P
|
|
><OL
|
|
TYPE="1"
|
|
><LI
|
|
><P
|
|
> Get all of the
|
|
<A
|
|
HREF="http://jabberd.jabberstudio.org/"
|
|
TARGET="_top"
|
|
>source code for jabberd14</A
|
|
>.
|
|
Build the server, configure/make.
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
> Get the
|
|
<A
|
|
HREF="http://download.jabber.org/contrib/xdb_sql-1.2.tar.gz"
|
|
TARGET="_top"
|
|
>source</A
|
|
>
|
|
for xdb_sql from
|
|
<A
|
|
HREF="http://www.IDEALX.org"
|
|
TARGET="_top"
|
|
>IDEALX</A
|
|
>
|
|
, build it, and setup the jabber.xml to use it.
|
|
</P
|
|
><DIV
|
|
CLASS="note"
|
|
><P
|
|
></P
|
|
><TABLE
|
|
CLASS="note"
|
|
WIDTH="100%"
|
|
BORDER="0"
|
|
><TR
|
|
><TD
|
|
WIDTH="25"
|
|
ALIGN="CENTER"
|
|
VALIGN="TOP"
|
|
><IMG
|
|
SRC="../images/note.gif"
|
|
HSPACE="5"
|
|
ALT="Note"></TD
|
|
><TD
|
|
ALIGN="LEFT"
|
|
VALIGN="TOP"
|
|
><P
|
|
> This is a very important step. xdb_file,
|
|
the default xdb that comes with jabberd,
|
|
is limited to open file descriptors too.
|
|
You can play the same shell games that we
|
|
are going to play with jadc2s later, but
|
|
if you want a server that can handle
|
|
millions or users, then you need something
|
|
other than xdb_file. Enter xdb_sql, which
|
|
only uses one file descriptor to connect
|
|
to the mysql server.
|
|
</P
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
></DIV
|
|
><P
|
|
> For more information on how to configure
|
|
xdb_sql, please see the README that is
|
|
distributed in the release.
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
> Build jadc2s and distribute the binaries
|
|
to the boxes where they will run.
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
> Setup the main jabberd to accept the jadc2s
|
|
component connections. In the jabber.xml
|
|
config file, add the following XML:
|
|
</P
|
|
><TABLE
|
|
BORDER="0"
|
|
BGCOLOR="#E0E0E0"
|
|
WIDTH="100%"
|
|
><TR
|
|
><TD
|
|
><FONT
|
|
COLOR="#000000"
|
|
><PRE
|
|
CLASS="screen"
|
|
> <service id="jadc2s-1">
|
|
<accept>
|
|
<ip/>
|
|
<port>5111</port>
|
|
<secret>secret</secret>
|
|
</accept>
|
|
</service>
|
|
|
|
<service id="jadc2s-2">
|
|
<accept>
|
|
<ip/>
|
|
<port>5112</port>
|
|
<secret>secret</secret>
|
|
</accept>
|
|
</service>
|
|
</PRE
|
|
></FONT
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><P
|
|
> Now you can run the main jabberd and get it
|
|
listening for the jadc2s to connect to
|
|
it.
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
> Configure the jadc2s.xml on each box to
|
|
connect to the SM, where to listen, etc...
|
|
</P
|
|
><TABLE
|
|
BORDER="0"
|
|
BGCOLOR="#E0E0E0"
|
|
WIDTH="100%"
|
|
><TR
|
|
><TD
|
|
><FONT
|
|
COLOR="#000000"
|
|
><PRE
|
|
CLASS="screen"
|
|
> <!-- session manager configuration -->
|
|
<sm>
|
|
<!-- host and port of the SM -->
|
|
<host>localhost</host>
|
|
<port>5111</port>
|
|
<!-- shared secret, for authenticating us -->
|
|
<secret>secret</secret>
|
|
|
|
<!-- our ID, for authenticating us to the sm -->
|
|
<id>jadc2s</id>
|
|
|
|
<!-- how many times to try to connect to the sm (default: 5) -->
|
|
<retries>5</retries>
|
|
</sm>
|
|
|
|
<!-- local network settings -->
|
|
<local>
|
|
<!-- who we identify ourselves as. This should correspond to the -->
|
|
<!-- ID (host) that the session manager thinks it is. -->
|
|
<id>localhost</id>
|
|
|
|
<!-- IP address to bind to (default: 0.0.0.0) -->
|
|
<!-- <ip>0.0.0.0</ip> -->
|
|
|
|
<!-- port to bind to (default: 5222) -->
|
|
<port>5222</port>
|
|
|
|
<!-- SSL configuration -->
|
|
<!-- Specify the port to listen on, and the key file to use for -->
|
|
<!-- the certificate. -->
|
|
<!-- <port/> (default: 5223) -->
|
|
<!-- <pemfile/> (default: ./server.pem) -->
|
|
<!--
|
|
<ssl>
|
|
<port>5223</port>
|
|
<pemfile>./server.pem</pemfile>
|
|
</ssl>
|
|
-->
|
|
</local>
|
|
</PRE
|
|
></FONT
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><P
|
|
> For more information on how to configure
|
|
jadc2s, please see the README in the jadc2s
|
|
source directory.
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
> Open a shell where you can change file
|
|
system parameters (root usually) and execute
|
|
the following command:
|
|
</P
|
|
><DIV
|
|
CLASS="informalexample"
|
|
><A
|
|
NAME="AEN108"><P
|
|
></P
|
|
><TABLE
|
|
BORDER="0"
|
|
BGCOLOR="#E0E0E0"
|
|
WIDTH="100%"
|
|
><TR
|
|
><TD
|
|
><FONT
|
|
COLOR="#000000"
|
|
><PRE
|
|
CLASS="screen"
|
|
> <TT
|
|
CLASS="prompt"
|
|
>bash$</TT
|
|
> <B
|
|
CLASS="command"
|
|
>echo "24000" > /proc/sys/fs/file-max</B
|
|
>
|
|
</PRE
|
|
></FONT
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><P
|
|
></P
|
|
></DIV
|
|
><P
|
|
> This bumps the upper bound on the
|
|
number of allowed file descriptors
|
|
that can be open at one time.
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
> Set the limit for the shell you are in
|
|
to use more than the default 1024 file
|
|
descriptors.
|
|
</P
|
|
><DIV
|
|
CLASS="informalexample"
|
|
><A
|
|
NAME="AEN115"><P
|
|
></P
|
|
><TABLE
|
|
BORDER="0"
|
|
BGCOLOR="#E0E0E0"
|
|
WIDTH="100%"
|
|
><TR
|
|
><TD
|
|
><FONT
|
|
COLOR="#000000"
|
|
><PRE
|
|
CLASS="screen"
|
|
> <TT
|
|
CLASS="prompt"
|
|
>bash$</TT
|
|
> <B
|
|
CLASS="command"
|
|
>ulimit -n 11000</B
|
|
>
|
|
</PRE
|
|
></FONT
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><P
|
|
></P
|
|
></DIV
|
|
></LI
|
|
><LI
|
|
><P
|
|
> Tell jadc2s how many file descriptors it
|
|
is allowed to use:
|
|
</P
|
|
><TABLE
|
|
BORDER="0"
|
|
BGCOLOR="#E0E0E0"
|
|
WIDTH="100%"
|
|
><TR
|
|
><TD
|
|
><FONT
|
|
COLOR="#000000"
|
|
><PRE
|
|
CLASS="screen"
|
|
> <!-- maximum number of file descriptors. Should be a prime -->
|
|
<!-- number. At least four will be used by jadc2s itself, -->
|
|
<!-- usually around six or seven (default: 1023) -->
|
|
<!-- For a list of prime numbers: -->
|
|
<!-- http://www.utm.edu/research/primes/lists/small/10000.txt -->
|
|
<max_fds>10007</max_fds>
|
|
</PRE
|
|
></FONT
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><P
|
|
> It is important that the number you pick is
|
|
a prime number. To make it easy to find the
|
|
prime you want you can visit this page
|
|
<A
|
|
HREF="http://www.utm.edu/research/primes/lists/small/10000.txt"
|
|
TARGET="_top"
|
|
>http://www.utm.edu/research/primes/lists/small/10000.txt</A
|
|
>.
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
> All that's left is to run the server, and the
|
|
jadc2s processes. Everything should work fine.
|
|
If it doesn't, then PLEASE tell me at
|
|
<TT
|
|
CLASS="email"
|
|
><<A
|
|
HREF="mailto:reatmon@jabber.org"
|
|
>reatmon@jabber.org</A
|
|
>></TT
|
|
> so that I can
|
|
fix this document.
|
|
</P
|
|
></LI
|
|
></OL
|
|
></DIV
|
|
></DIV
|
|
><DIV
|
|
CLASS="NAVFOOTER"
|
|
><HR
|
|
ALIGN="LEFT"
|
|
WIDTH="100%"><TABLE
|
|
SUMMARY="Footer navigation table"
|
|
WIDTH="100%"
|
|
BORDER="0"
|
|
CELLPADDING="0"
|
|
CELLSPACING="0"
|
|
><TR
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="left"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="implementation.html"
|
|
ACCESSKEY="P"
|
|
>Prev</A
|
|
></TD
|
|
><TD
|
|
WIDTH="34%"
|
|
ALIGN="center"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="index.html"
|
|
ACCESSKEY="H"
|
|
>Home</A
|
|
></TD
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="right"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="redirection.html"
|
|
ACCESSKEY="N"
|
|
>Next</A
|
|
></TD
|
|
></TR
|
|
><TR
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="left"
|
|
VALIGN="top"
|
|
>Implementing the Farm</TD
|
|
><TD
|
|
WIDTH="34%"
|
|
ALIGN="center"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="implementation.html"
|
|
ACCESSKEY="U"
|
|
>Up</A
|
|
></TD
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="right"
|
|
VALIGN="top"
|
|
>Connection Redirection</TD
|
|
></TR
|
|
></TABLE
|
|
></DIV
|
|
></BODY
|
|
></HTML
|
|
> |