542 lines
11 KiB
HTML
542 lines
11 KiB
HTML
<HTML
|
|
><HEAD
|
|
><TITLE
|
|
>Monitoring and administration</TITLE
|
|
><META
|
|
NAME="GENERATOR"
|
|
CONTENT="Modular DocBook HTML Stylesheet Version 1.76b+
|
|
"><LINK
|
|
REL="HOME"
|
|
TITLE="Usenet News HOWTO "
|
|
HREF="index.html"><LINK
|
|
REL="PREVIOUS"
|
|
TITLE="Components of a running system"
|
|
HREF="component.html"><LINK
|
|
REL="NEXT"
|
|
TITLE="Usenet news clients"
|
|
HREF="x1208.html"></HEAD
|
|
><BODY
|
|
CLASS="SECTION"
|
|
BGCOLOR="#FFFFFF"
|
|
TEXT="#000000"
|
|
LINK="#0000FF"
|
|
VLINK="#840084"
|
|
ALINK="#0000FF"
|
|
><DIV
|
|
CLASS="NAVHEADER"
|
|
><TABLE
|
|
SUMMARY="Header navigation table"
|
|
WIDTH="100%"
|
|
BORDER="0"
|
|
CELLPADDING="0"
|
|
CELLSPACING="0"
|
|
><TR
|
|
><TH
|
|
COLSPAN="3"
|
|
ALIGN="center"
|
|
>Usenet News HOWTO</TH
|
|
></TR
|
|
><TR
|
|
><TD
|
|
WIDTH="10%"
|
|
ALIGN="left"
|
|
VALIGN="bottom"
|
|
><A
|
|
HREF="component.html"
|
|
ACCESSKEY="P"
|
|
>Prev</A
|
|
></TD
|
|
><TD
|
|
WIDTH="80%"
|
|
ALIGN="center"
|
|
VALIGN="bottom"
|
|
></TD
|
|
><TD
|
|
WIDTH="10%"
|
|
ALIGN="right"
|
|
VALIGN="bottom"
|
|
><A
|
|
HREF="x1208.html"
|
|
ACCESSKEY="N"
|
|
>Next</A
|
|
></TD
|
|
></TR
|
|
></TABLE
|
|
><HR
|
|
ALIGN="LEFT"
|
|
WIDTH="100%"></DIV
|
|
><DIV
|
|
CLASS="SECTION"
|
|
><H1
|
|
CLASS="SECTION"
|
|
><A
|
|
NAME="AEN1094">10. Monitoring and administration</H1
|
|
><P
|
|
>Once the Usenet News system is in place and running, the news administrator
|
|
is then aided in monitoring the system by various reports generated by it.
|
|
Also, he needs to make regular checks in specific directories and
|
|
files to ascertain the smooth working of the system.</P
|
|
><DIV
|
|
CLASS="SECTION"
|
|
><H2
|
|
CLASS="SECTION"
|
|
><A
|
|
NAME="AEN1097">10.1. The <TT
|
|
CLASS="LITERAL"
|
|
>newsdaily</TT
|
|
> report</H2
|
|
><P
|
|
>This report is generated by the script <TT
|
|
CLASS="LITERAL"
|
|
>newsdaily</TT
|
|
> which is
|
|
typically run through <TT
|
|
CLASS="LITERAL"
|
|
>cron</TT
|
|
>. I shall enumerate some of the
|
|
problems that are reported by it, based on my observations .</P
|
|
><P
|
|
></P
|
|
><UL
|
|
><LI
|
|
><P
|
|
><EM
|
|
>bad input batches</EM
|
|
>:
|
|
This gives a list of articles that have been declared bad after processing
|
|
and hence have not been digested. The reason for this is not given. You
|
|
are expected to check each article and determine the cause. </P
|
|
></LI
|
|
><LI
|
|
><P
|
|
><EM
|
|
>leading unknown newsgroups by articles</EM
|
|
>:
|
|
Newsgroup names that do not appear in the <TT
|
|
CLASS="LITERAL"
|
|
>active</TT
|
|
> file
|
|
but their hierarchy has been subscribed to, would find their names mentioned
|
|
under this heading. Choose to add the name in the active file if you think
|
|
it is important. For <EM
|
|
>e.g.</EM
|
|
>, you would see this happen
|
|
if you have subscribed to the hierarchy <TT
|
|
CLASS="LITERAL"
|
|
>comp</TT
|
|
> but the
|
|
<TT
|
|
CLASS="LITERAL"
|
|
>active</TT
|
|
> does not contain the newsgroup name
|
|
<TT
|
|
CLASS="LITERAL"
|
|
>comp.lang.java.3d</TT
|
|
>. You could deny subscription to this
|
|
particular newsgroup by specifying so in the <TT
|
|
CLASS="LITERAL"
|
|
>sys</TT
|
|
> file.
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
><EM
|
|
>leading unsubscribed newsgroups</EM
|
|
>:
|
|
If the news server receives maximum articles of a particular newsgroup
|
|
hierarchy to which you haven't subscribed, it will appear under this
|
|
heading. You really cannot do much about this except to
|
|
subscribe to them if they are required.</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
><EM
|
|
>leading sites sending bad headers</EM
|
|
>:
|
|
This will list your NDNs who
|
|
are sending articles with malformed/insufficient headers. </P
|
|
></LI
|
|
><LI
|
|
><P
|
|
><EM
|
|
>leading sites sending stale/future/misdated news</EM
|
|
>:
|
|
This will list your NDNs who are sending you articles that are older than
|
|
the date you have specified for accepting feeds. </P
|
|
></LI
|
|
><LI
|
|
><P
|
|
>Some of the reports generated by us:
|
|
We have modified the newsdaily script to include some more statistics.
|
|
<P
|
|
></P
|
|
><UL
|
|
><LI
|
|
><P
|
|
><EM
|
|
>disk usage</EM
|
|
>:
|
|
This reports the size in bytes of the <TT
|
|
CLASS="LITERAL"
|
|
>$NEWSARTS</TT
|
|
>
|
|
area. If you are receiving feeds regularly, you should see this figure
|
|
increasing.
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
><EM
|
|
>incoming feed statistics</EM
|
|
>:
|
|
This reports the number of articles and total bytes recevied from each
|
|
of your NDNs.
|
|
</P
|
|
></LI
|
|
><LI
|
|
><P
|
|
><EM
|
|
>NNTP traffic report</EM
|
|
>:
|
|
The output of nestor has also been included in this report which gives
|
|
details of each <TT
|
|
CLASS="LITERAL"
|
|
>nntp</TT
|
|
> connection and the overall
|
|
performance of the network connection read from the <TT
|
|
CLASS="LITERAL"
|
|
>newslog
|
|
</TT
|
|
> file. To understand the format, read the manpage of
|
|
<TT
|
|
CLASS="LITERAL"
|
|
>nestor</TT
|
|
>.
|
|
</P
|
|
></LI
|
|
></UL
|
|
></P
|
|
></LI
|
|
><LI
|
|
><P
|
|
><EM
|
|
>Error reporting from the <TT
|
|
CLASS="LITERAL"
|
|
>errorlog</TT
|
|
>
|
|
file</EM
|
|
>:
|
|
Reports errors logged in the <TT
|
|
CLASS="LITERAL"
|
|
>errorlog</TT
|
|
> file. Usually
|
|
these are file ownership or file missing problems which can be easily
|
|
handled. </P
|
|
></LI
|
|
></UL
|
|
></DIV
|
|
><DIV
|
|
CLASS="SECTION"
|
|
><H2
|
|
CLASS="SECTION"
|
|
><A
|
|
NAME="AEN1146">10.2. Crisis reports from <TT
|
|
CLASS="LITERAL"
|
|
>newswatch</TT
|
|
></H2
|
|
><P
|
|
>Most of the problems reported to me are those with either space shortage or
|
|
persistent locks. There are instances when the scripts have created locks files
|
|
and have aborted/terminated without removing them. Sometimes they are
|
|
innocuous enough to be deleted but this should be determined after a careful
|
|
analysis. They could be an indication of some part of the system not working
|
|
correctly. For <EM
|
|
>e.g.</EM
|
|
> I would receive this error message when
|
|
sendbatches would abnormally terminate trying to transmit huge togo files. I had
|
|
to determine why sendbatches was failing this often.</P
|
|
><P
|
|
>The space shortage issue has to be addressed immediately. You could
|
|
delete unwanted articles by running <TT
|
|
CLASS="LITERAL"
|
|
>doexpire</TT
|
|
> or add more disk
|
|
space at the OS level. </P
|
|
></DIV
|
|
><DIV
|
|
CLASS="SECTION"
|
|
><H2
|
|
CLASS="SECTION"
|
|
><A
|
|
NAME="AEN1153">10.3. Disk space</H2
|
|
><P
|
|
>The <TT
|
|
CLASS="LITERAL"
|
|
>$NEWSBIN</TT
|
|
> area occupies space that is fixed. Since the
|
|
binaries do not grow once installed, you do not have to worry about disk
|
|
shortage here. The areas that take up more space as feeds come in are
|
|
<TT
|
|
CLASS="LITERAL"
|
|
>$NEWSCTL</TT
|
|
> and <TT
|
|
CLASS="LITERAL"
|
|
>$NEWSARTS</TT
|
|
>. The
|
|
<TT
|
|
CLASS="LITERAL"
|
|
>$NEWSCTL</TT
|
|
> has log files that keep growing with each feed.
|
|
As the articles are digested in huge numbers, the <TT
|
|
CLASS="LITERAL"
|
|
>$NEWSARTS</TT
|
|
>
|
|
area continues to grow. Also, you will need space if you have chosen to archive
|
|
articles on expiry. Allocate a few GB of disk space for
|
|
<TT
|
|
CLASS="LITERAL"
|
|
>$NEWSARTS</TT
|
|
> depending on the number of hierarchies you are
|
|
subscribing and the feeds that come in everyday. <TT
|
|
CLASS="LITERAL"
|
|
>$NEWSCTL</TT
|
|
>
|
|
grows to a lesser proportion as compared to <TT
|
|
CLASS="LITERAL"
|
|
>$NEWSARTS</TT
|
|
>.
|
|
Allocate space for this accordingly.</P
|
|
></DIV
|
|
><DIV
|
|
CLASS="SECTION"
|
|
><H2
|
|
CLASS="SECTION"
|
|
><A
|
|
NAME="AEN1164">10.4. CPU load and RAM usage</H2
|
|
><P
|
|
>With modern C-News and NNTPd, there is very little usage of these
|
|
system resources for processing news article flow. Key components like
|
|
<TT
|
|
CLASS="LITERAL"
|
|
>newsrun</TT
|
|
> or <TT
|
|
CLASS="LITERAL"
|
|
>sendbatches</TT
|
|
> do not load
|
|
the system much, except for cases where you have a very heavy flow of
|
|
compressed outgoing batches and the compression utility is run by
|
|
<TT
|
|
CLASS="LITERAL"
|
|
>sendbatches</TT
|
|
> frequently. <TT
|
|
CLASS="LITERAL"
|
|
>newsrun</TT
|
|
> is
|
|
amazingly efficient in the current C-News release. Even when it takes
|
|
half an hour to digest a large consignment of batches, it hardly loads the
|
|
CPU of a slow Pentium 200 MHz CPU or consumes much RAM in a 64 MB
|
|
system.</P
|
|
><P
|
|
>One thing which does slow down a system is a large bunch of
|
|
users connecting using NNTP to browse newsgroups. We do not have
|
|
heuristic based figures off-hand to provide a guidance figure for
|
|
resource consumption for this, but we have found that the load on the
|
|
CPU and RAM for a certain number of active users invoking
|
|
<TT
|
|
CLASS="LITERAL"
|
|
>nntpd</TT
|
|
> is more than with an equal number of
|
|
users connecting to the POP3 port of the same system for pulling
|
|
out mailboxes. A few hundred active NNTP users can really slow down
|
|
a dual-P-III Intel Linux server, for instance. This loading has no
|
|
bearing on whether you are using INN or <TT
|
|
CLASS="LITERAL"
|
|
>nntpd</TT
|
|
>;
|
|
both have practically identical implementations for NNTP
|
|
<EM
|
|
>reading</EM
|
|
> and differ only in their handling of
|
|
feeds.</P
|
|
><P
|
|
>Another situation which will slow down your Usenet news server is
|
|
when downstream servers connect to you for pulling out NNTP feeds using
|
|
the pull method. This has been mentioned before. This can really load
|
|
your server's I/O system and CPU.</P
|
|
></DIV
|
|
><DIV
|
|
CLASS="SECTION"
|
|
><H2
|
|
CLASS="SECTION"
|
|
><A
|
|
NAME="AEN1176">10.5. The <TT
|
|
CLASS="LITERAL"
|
|
>in.coming/bad</TT
|
|
> directory</H2
|
|
><P
|
|
>The <TT
|
|
CLASS="LITERAL"
|
|
>in.coming</TT
|
|
> directory is where the batches/articles reside when you have
|
|
received feeds from your NDN and before processing happens. Checking this
|
|
directory regularly to see if there are batches is a good way of determining
|
|
that feeds are coming in. The batches and articles have different nomenclature.
|
|
Batches, typically, have names like <EM
|
|
>nntp.GxhsDj</EM
|
|
> and
|
|
individual articles are named beginning with digits like <EM
|
|
>0.10022643380.t</EM
|
|
></P
|
|
><P
|
|
>The <TT
|
|
CLASS="LITERAL"
|
|
>bad</TT
|
|
> sub-directory under <TT
|
|
CLASS="LITERAL"
|
|
>in.coming</TT
|
|
>
|
|
holds batches/articles that have encountered errors when they were being
|
|
processed by <TT
|
|
CLASS="LITERAL"
|
|
>relaynews</TT
|
|
>. You will have to look at the
|
|
individual files in this directory to determine the cause . Ideally speaking,
|
|
this directory should be empty.</P
|
|
></DIV
|
|
><DIV
|
|
CLASS="SECTION"
|
|
><H2
|
|
CLASS="SECTION"
|
|
><A
|
|
NAME="AEN1187">10.6. Long pending queues in <TT
|
|
CLASS="LITERAL"
|
|
>out.going</TT
|
|
></H2
|
|
><P
|
|
>TO BE ADDED.</P
|
|
></DIV
|
|
><DIV
|
|
CLASS="SECTION"
|
|
><H2
|
|
CLASS="SECTION"
|
|
><A
|
|
NAME="AEN1191">10.7. Problems with <TT
|
|
CLASS="LITERAL"
|
|
>nntpxmit</TT
|
|
> and <TT
|
|
CLASS="LITERAL"
|
|
>nntpsend</TT
|
|
></H2
|
|
><P
|
|
>TO BE ADDED.</P
|
|
></DIV
|
|
><DIV
|
|
CLASS="SECTION"
|
|
><H2
|
|
CLASS="SECTION"
|
|
><A
|
|
NAME="AEN1196">10.8. The <TT
|
|
CLASS="LITERAL"
|
|
>junk</TT
|
|
> and <TT
|
|
CLASS="LITERAL"
|
|
>control</TT
|
|
> groups</H2
|
|
><P
|
|
>Control messages are those that have a <TT
|
|
CLASS="LITERAL"
|
|
>newgroup/rmgroup/cancel/checkgroup</TT
|
|
> in
|
|
their subject line. Such messages result in <TT
|
|
CLASS="LITERAL"
|
|
>relaynews</TT
|
|
> calling
|
|
the appropriate script and on execution a message is mailed to the admin about
|
|
the action taken. These control messages are stored in the
|
|
<TT
|
|
CLASS="LITERAL"
|
|
>control</TT
|
|
> directory of <TT
|
|
CLASS="LITERAL"
|
|
>$NEWSARTS</TT
|
|
>. For the
|
|
propogation of such messages, one must subscribe to the
|
|
<TT
|
|
CLASS="LITERAL"
|
|
>control</TT
|
|
> hierarchy.</P
|
|
><P
|
|
>When your news system determines that a certain article has not been subscribed
|
|
by you, it is <SPAN
|
|
CLASS="QUOTE"
|
|
>"junked"</SPAN
|
|
> i.e. such articles appear in the junk
|
|
directory. This
|
|
directory plays a key role in transferring articles to your NDNs as they would
|
|
subscribe to the junk hierarchy to receive feeds. If you are a leaf node, there
|
|
is no reason why articles should pile here. Keep deleting them on a daily
|
|
basis.</P
|
|
></DIV
|
|
></DIV
|
|
><DIV
|
|
CLASS="NAVFOOTER"
|
|
><HR
|
|
ALIGN="LEFT"
|
|
WIDTH="100%"><TABLE
|
|
SUMMARY="Footer navigation table"
|
|
WIDTH="100%"
|
|
BORDER="0"
|
|
CELLPADDING="0"
|
|
CELLSPACING="0"
|
|
><TR
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="left"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="component.html"
|
|
ACCESSKEY="P"
|
|
>Prev</A
|
|
></TD
|
|
><TD
|
|
WIDTH="34%"
|
|
ALIGN="center"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="index.html"
|
|
ACCESSKEY="H"
|
|
>Home</A
|
|
></TD
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="right"
|
|
VALIGN="top"
|
|
><A
|
|
HREF="x1208.html"
|
|
ACCESSKEY="N"
|
|
>Next</A
|
|
></TD
|
|
></TR
|
|
><TR
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="left"
|
|
VALIGN="top"
|
|
>Components of a running system</TD
|
|
><TD
|
|
WIDTH="34%"
|
|
ALIGN="center"
|
|
VALIGN="top"
|
|
> </TD
|
|
><TD
|
|
WIDTH="33%"
|
|
ALIGN="right"
|
|
VALIGN="top"
|
|
>Usenet news clients</TD
|
|
></TR
|
|
></TABLE
|
|
></DIV
|
|
></BODY
|
|
></HTML
|
|
> |