mirror of https://github.com/tLDP/LDP
211 lines
9.3 KiB
Plaintext
211 lines
9.3 KiB
Plaintext
<section><title>Monitoring and administration</title>
|
|
<para>
|
|
Once the Usenet News system is in place and running, the news administrator
|
|
is then aided in monitoring the system by various reports generated by it.
|
|
Also, he needs to make regular checks in specific directories and
|
|
files to ascertain the smooth working of the system.
|
|
</para>
|
|
|
|
<section><title>The <literal>newsdaily</literal> report</title>
|
|
<para>
|
|
This report is generated by the script <literal>newsdaily</literal> which is
|
|
typically run through <literal>cron</literal>. I shall enumerate some of the
|
|
problems that are reported by it, based on my observations .</para>
|
|
|
|
<itemizedlist>
|
|
<listitem><para><emphasis>bad input batches</emphasis>:
|
|
This gives a list of articles that have been declared bad after processing
|
|
and hence have not been digested. The reason for this is not given. You
|
|
are expected to check each article and determine the cause.
|
|
</para></listitem>
|
|
|
|
<listitem><para><emphasis>leading unknown newsgroups by articles</emphasis>:
|
|
Newsgroup names that do not appear in the <literal>active</literal> file
|
|
but their hierarchy has been subscribed to, would find their names mentioned
|
|
under this heading. Choose to add the name in the active file if you think
|
|
it is important. For <emphasis>e.g.</emphasis>, you would see this happen
|
|
if you have subscribed to the hierarchy <literal>comp</literal> but the
|
|
<literal>active</literal> does not contain the newsgroup name
|
|
<literal>comp.lang.java.3d</literal>. You could deny subscription to this
|
|
particular newsgroup by specifying so in the <literal>sys</literal> file.
|
|
|
|
</para></listitem>
|
|
|
|
<listitem><para><emphasis>leading unsubscribed newsgroups</emphasis>:
|
|
If the news server receives maximum articles of a particular newsgroup
|
|
hierarchy to which you haven't subscribed, it will appear under this
|
|
heading. You really cannot do much about this except to
|
|
subscribe to them if they are required.
|
|
</para></listitem>
|
|
|
|
<listitem><para><emphasis>leading sites sending bad headers</emphasis>:
|
|
This will list your NDNs who
|
|
are sending articles with malformed/insufficient headers.
|
|
</para></listitem>
|
|
|
|
<listitem><para><emphasis>leading sites sending stale/future/misdated news</emphasis>:
|
|
This will list your NDNs who are sending you articles that are older than
|
|
the date you have specified for accepting feeds.
|
|
</para></listitem>
|
|
|
|
<listitem><para>Some of the reports generated by us:
|
|
We have modified the newsdaily script to include some more statistics.
|
|
<itemizedlist>
|
|
<listitem><para><emphasis>disk usage</emphasis>:
|
|
This reports the size in bytes of the <literal>$NEWSARTS</literal>
|
|
area. If you are receiving feeds regularly, you should see this figure
|
|
increasing.
|
|
</para></listitem>
|
|
|
|
<listitem><para><emphasis>incoming feed statistics</emphasis>:
|
|
This reports the number of articles and total bytes recevied from each
|
|
of your NDNs.
|
|
</para></listitem>
|
|
|
|
<listitem><para><emphasis>NNTP traffic report</emphasis>:
|
|
The output of nestor has also been included in this report which gives
|
|
details of each <literal>nntp</literal> connection and the overall
|
|
performance of the network connection read from the <literal>newslog
|
|
</literal> file. To understand the format, read the manpage of
|
|
<literal>nestor</literal>.
|
|
</para></listitem>
|
|
</itemizedlist>
|
|
</para></listitem>
|
|
|
|
<listitem><para><emphasis>Error reporting from the <literal>errorlog</literal>
|
|
file</emphasis>:
|
|
Reports errors logged in the <literal>errorlog</literal> file. Usually
|
|
these are file ownership or file missing problems which can be easily
|
|
handled.
|
|
</para></listitem>
|
|
</itemizedlist>
|
|
</section>
|
|
|
|
<section><title>Crisis reports from <literal>newswatch</literal></title>
|
|
<para>
|
|
Most of the problems reported to me are those with either space shortage or
|
|
persistent locks. There are instances when the scripts have created locks files
|
|
and have aborted/terminated without removing them. Sometimes they are
|
|
innocuous enough to be deleted but this should be determined after a careful
|
|
analysis. They could be an indication of some part of the system not working
|
|
correctly. For <emphasis>e.g.</emphasis> I would receive this error message when
|
|
sendbatches would abnormally terminate trying to transmit huge togo files. I had
|
|
to determine why sendbatches was failing this often.
|
|
</para>
|
|
|
|
<para>
|
|
The space shortage issue has to be addressed immediately. You could
|
|
delete unwanted articles by running <literal>doexpire</literal> or add more disk
|
|
space at the OS level.
|
|
</para>
|
|
</section>
|
|
|
|
<section><title>Disk space</title>
|
|
<para>
|
|
The <literal>$NEWSBIN</literal> area occupies space that is fixed. Since the
|
|
binaries do not grow once installed, you do not have to worry about disk
|
|
shortage here. The areas that take up more space as feeds come in are
|
|
<literal>$NEWSCTL</literal> and <literal>$NEWSARTS</literal>. The
|
|
<literal>$NEWSCTL</literal> has log files that keep growing with each feed.
|
|
As the articles are digested in huge numbers, the <literal>$NEWSARTS</literal>
|
|
area continues to grow. Also, you will need space if you have chosen to archive
|
|
articles on expiry. Allocate a few GB of disk space for
|
|
<literal>$NEWSARTS</literal> depending on the number of hierarchies you are
|
|
subscribing and the feeds that come in everyday. <literal>$NEWSCTL</literal>
|
|
grows to a lesser proportion as compared to <literal>$NEWSARTS</literal>.
|
|
Allocate space for this accordingly.
|
|
</para>
|
|
</section>
|
|
|
|
<section><title>CPU load and RAM usage</title>
|
|
<para>
|
|
With modern C-News and NNTPd, there is very little usage of these
|
|
system resources for processing news article flow. Key components like
|
|
<literal>newsrun</literal> or <literal>sendbatches</literal> do not load
|
|
the system much, except for cases where you have a very heavy flow of
|
|
compressed outgoing batches and the compression utility is run by
|
|
<literal>sendbatches</literal> frequently. <literal>newsrun</literal> is
|
|
amazingly efficient in the current C-News release. Even when it takes
|
|
half an hour to digest a large consignment of batches, it hardly loads the
|
|
CPU of a slow Pentium 200 MHz CPU or consumes much RAM in a 64 MB
|
|
system.
|
|
</para>
|
|
|
|
<para>
|
|
One thing which does slow down a system is a large bunch of
|
|
users connecting using NNTP to browse newsgroups. We do not have
|
|
heuristic based figures off-hand to provide a guidance figure for
|
|
resource consumption for this, but we have found that the load on the
|
|
CPU and RAM for a certain number of active users invoking
|
|
<literal>nntpd</literal> is more than with an equal number of
|
|
users connecting to the POP3 port of the same system for pulling
|
|
out mailboxes. A few hundred active NNTP users can really slow down
|
|
a dual-P-III Intel Linux server, for instance. This loading has no
|
|
bearing on whether you are using INN or <literal>nntpd</literal>;
|
|
both have practically identical implementations for NNTP
|
|
<emphasis>reading</emphasis> and differ only in their handling of
|
|
feeds.
|
|
</para>
|
|
|
|
<para>Another situation which will slow down your Usenet news server is
|
|
when downstream servers connect to you for pulling out NNTP feeds using
|
|
the pull method. This has been mentioned before. This can really load
|
|
your server's I/O system and CPU.</para>
|
|
|
|
</section>
|
|
|
|
<section><title>The <literal>in.coming/bad</literal> directory</title>
|
|
<para>
|
|
The <literal>in.coming</literal> directory is where the batches/articles reside when you have
|
|
received feeds from your NDN and before processing happens. Checking this
|
|
directory regularly to see if there are batches is a good way of determining
|
|
that feeds are coming in. The batches and articles have different nomenclature.
|
|
Batches, typically, have names like <emphasis>nntp.GxhsDj</emphasis> and
|
|
individual articles are named beginning with digits like <emphasis>0.10022643380.t</emphasis>
|
|
</para>
|
|
|
|
<para>
|
|
The <literal>bad</literal> sub-directory under <literal>in.coming</literal>
|
|
holds batches/articles that have encountered errors when they were being
|
|
processed by <literal>relaynews</literal>. You will have to look at the
|
|
individual files in this directory to determine the cause . Ideally speaking,
|
|
this directory should be empty.
|
|
</para>
|
|
</section>
|
|
|
|
<section><title>Long pending queues in <literal>out.going</literal></title>
|
|
|
|
<para>TO BE ADDED.</para>
|
|
|
|
</section>
|
|
|
|
<section><title>Problems with <literal>nntpxmit</literal> and <literal>nntpsend</literal></title>
|
|
|
|
<para>TO BE ADDED.</para>
|
|
|
|
</section>
|
|
|
|
<section><title>The <literal>junk</literal> and <literal>control</literal> groups</title>
|
|
<para>
|
|
Control messages are those that have a <literal>newgroup/rmgroup/cancel/checkgroup</literal> in
|
|
their subject line. Such messages result in <literal>relaynews</literal> calling
|
|
the appropriate script and on execution a message is mailed to the admin about
|
|
the action taken. These control messages are stored in the
|
|
<literal>control</literal> directory of <literal>$NEWSARTS</literal>. For the
|
|
propogation of such messages, one must subscribe to the
|
|
<literal>control</literal> hierarchy.
|
|
</para>
|
|
|
|
<para>
|
|
When your news system determines that a certain article has not been subscribed
|
|
by you, it is <quote>junked</quote> i.e. such articles appear in the junk
|
|
directory. This
|
|
directory plays a key role in transferring articles to your NDNs as they would
|
|
subscribe to the junk hierarchy to receive feeds. If you are a leaf node, there
|
|
is no reason why articles should pile here. Keep deleting them on a daily
|
|
basis.
|
|
</para>
|
|
|
|
</section>
|
|
</section>
|