LDP
/
old-www
1
1
Fork 0
Code Issues Pull Requests Releases Wiki Activity
old-www/LDP/LG/issue09/newsserver.html

443 lines
14 KiB
HTML
Raw Permalink Blame History

<HTML>
<HEAD>
<TITLE>Easy local News server</TITLE>
</HEAD>
<BODY>
<H1 ALIGN=CENTER>The Easy Way to Set Up a Local News Server</H1>
<P ALIGN=CENTER>
By <STRONG> Christophe Blaess </STRONG>
(<A HREF=mailto:ccb@club-internet.fr>ccb@club-internet.fr</A>).
<H2>Introduction</H2>
</P>
<P>A few months ago, I decided to set up a local news server on my Linux Box,
in order to read off-line the articles. Before then I read
the news directly from the Usenet server of my Internet Provider, at the
detriment of my phone bill...(Here, in France, even the local communications
are rather expensive)</P>
<P>Before trying to install and set up one of the two classical news servers
(<EM>Inn</EM> and <EM>CNews</EM>), I browsed a bit the Web, looking for a
possible other product. I then discovered a very powerful small package
"<A HREF="http://www.troll.no/freebies/leafnode.html"><EM>Leafnode</EM></A>"
written by <A HREF="mailto:agulbra@troll.no">Arnt Gulbrandsen</A>.
It can be found in source form at <A
HREF="ftp://ftp.troll.no/freebies/leafnode/"><EM>ftp://ftp.troll.no/freebies/leafnode/</EM></A>
and the home page of this project is accessible at
<A HREF="http://www.troll.no/freebies/leafnode.html"><EM>http://www.troll.no/freebies/leafnode.html</EM></A>.</P>
<P>This package (leafnode-0.9.tar.gz, 29 Kb) contains three little programs,
very easy to install, and to use:</P>
<UL>
<LI>
"<STRONG>Fetch</STRONG>" can feed a local news server from a remote
Usenet server (university, ISP...).
It also posts the outgoing articles, using classical NNTP requests.
<P><LI>
"<STRONG>Leafnode</STRONG>" is a USENET server, run by the <EM>inetd</EM>
daemon, when a connection in required on the NNTP port, by a newsreader.
<P><LI>"<STRONG>Texpire</STRONG>" is generally run daily from the <EM>crontab</EM>
to erase the oldest articles from the news spool.</P>
</UL>
<P>The main advantage of this system, is the transparent way it is
inserted between the remote news server and the local newsreader. The
remote Usenet server sees
Leafnode exactly like a classical newsreader (like tin, trn,
netscape,...) and the local
news reader sees Leafnode just like a USENET server.</P>
<P>I have been happily using Leafnode for several months, and I would like to
describe here the (very simple) steps to <A HREF="#INSTALL">install</A> and
<A HREF="#CONFIGURE">configure</A> it. Then I will explain some
<A HREF="#HINTS">hints</A> to use it in a multi-users environment.</P>
<HR>
<H2><A NAME="INSTALL">INSTALLING LEAFNODE</A></H2>
<P>I've been using leafnode 0.8 for a few months, but I've recently upgraded
to version 0.9. The process for installing it is the same as the previous
version, but I've had a little problem, maybe due to my version of
<EM>make</EM>, and I'll describe how I have fixed it.</P>
<P>You first need to download the source code from
<A HREF="ftp://ftp.troll.no/freebies/leafnode/">ftp://ftp.troll.no/freebies/leafnode/</A>.
You'll get <STRONG>leafnode-0.9.tar.gz</STRONG>.</P>
<h3>Compiling the source code</h3>
<P>Become <EM>root</EM> and do :</P>
<PRE>
# cd /usr/local/src
# tar -xzf ~/leafnode-0.9.tar.gz
# cd leafnode-0.9/
# make
</PRE>
<P>Everything must compile without any problem...</P>
<h3>Installing leafnode</h3>
<P>Make sure there is a "<EM>news</EM>" user and a "<EM>news</EM>" group
on your system.</P>
<P>Then you can type:</P>
<PRE>
# make install
</PRE>
<P>If "<EM>make install</EM>" complains when making directories in
/var/spool/news/message.id/,
you may have the same problem as I've had. To fix it, I've modified the
Makefile, to insert a part of the leafnode-0.8 Makefile:</P>
<P>replace the lines (in the "install:" section)</P>
<PRE>
cd $(SPOOLDIR)/message.id
for a in 0 1 2 3 4 5 6 7 8 9 ; do for b in 0 1 2 3 4 5 6 7 8 9 ; do \
mkdir ${a}${b}0 ${a}${b}1 ${a}${b}2 ${a}${b}3 ${a}${b}4 ; \
mkdir ${a}${b}5 ${a}${b}6 ${a}${b}7 ${a}${b}8 ${a}${b}9 ; done
</PRE>
<P>by the line</P>
<PRE>
-mkdir -p $(SPOOLDIR)/message.id/{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}
</PRE>
<P>Be sure that the first character on the line is a tabulation. Then try again:</P>
<PRE>
# make install
</PRE>
<h3>End of installation</h3>
<P>At this point, Leafnode will have installed the following files on your
system:</P>
<PRE>
/usr/local/sbin/fetch
/usr/local/sbin/leafnode
/usr/local/sbin/texpire
/usr/local/man/fetch.8
/usr/local/man/leafnode.8
/usr/local/man/texpire.8
/usr/lib/leafnode/config.example
/var/spool/news/ ... and a lot of subdirectories ...
</PRE>
<P>First you have to copy the file /usr/lib/leafnode/config.example to
/usr/lib/leafnode/config and edit him, to put the name of your
remote NNTP server in place of:</P>
<PRE>
server = news.hiof.no
</PRE>
<P>Edit the file /etc/inetd.conf, and look for a line like:</P>
<PRE>
nntp stream tcp nowait root /usr/sbin/tcpd in.nntpd
</PRE>
<P>then modify it like this:</P>
<PRE>
nntp stream tcp nowait news /usr/sbin/tcpd /usr/local/sbin/leafnode
</PRE>
<P>Depending on your configuration, there may be no TCP wrapper installed,
so you could have to remove the "/usr/sbin/tcpd" part of this line.</P>
<P>and do</P>
<PRE>
# killall -HUP inetd
</PRE>
<H3>First run</H3>
<P>Connect to your Usenet provider, and, while being
"<EM>root</EM>" or "<EM>news</EM>", run:</P>
<PRE>
$ fetch
</PRE>
<P>You will have to wait for a moment, because Leafnode is asking the
NNTP server the list of all the active groups. Once fetch ends,
run a newsreader, as normal user, and ask him to contact the localhost.
for example, with <EM>tin</EM> do:</P>
<PRE>
$ export NNTPSERVER=localhost
$ tin -r
</PRE>
<P> (With Netscape you have to put <EM>localhost</EM> in "Options/Mail
and News/Servers/NNTP server")</P>
<P>You will get the list of all available newsgroups, then choose interesting
ones, and read them. They will appear empty at this time. It's normal.</P>
<P>As <EM>root</EM> run again fetch. It will download all the content of the
previously read newsgroups. The first downloading will take a while, but
the next will obviously be very much quicker.</P>
<P>Your local USENET server is installed!</P>
<hr>
<H2><A NAME="CONFIGURE">CONFIGURING LEAFNODE</A></H2>
<P>Now that leafnode run on your Linux box, you can configure some details:<P>
<P>In the file /usr/lib/leafnode/config, there are two fields you can edit:</P>
<UL>
<LI>the <EM>expire</EM> value tells to Texpire the number of days an
unread thread must be kept.
<LI>the <EM>maxcount</EM> value limits the number of articles to download
from a newsgroup, during a single execution of Fetch.
</UL>
<P>Refer to the comments in this file to choose the values (on my system I use
expire=10 and maxcount=3000)</P>
<P>You must run Texpire from time to time, and a daily entry in the
crontab seems to be the best choice:</P>
<PRE>
00 03 * * * news /usr/local/sbin/texpire
</PRE>
<P>This line is for a system-wide crontab (generally /etc/crontab
maintained by root),
but you will have to remove the username "<EM>news</EM>" on user crontab
(/var/spool/cron/news).</P>
<P>You will have to run periodically <EM>fetch</EM>. If you have a
permanent link with your news server, there's an obvious solution: the
crontab again, to run it once per hour for example.</P>
<PRE>
00 * * * * news /usr/local/sbin/fetch
</PRE>
<P>If you're using a non-permanent PPP connection, you can insert
<EM>fetch</EM> at the end of the ip-up shell script (see Linux Gazette 7
"<CITE>Setting up PPP's ip-up and ip-down scripts!</CITE>").</P>
<P>Put the name "<CODE>localhost</CODE>" in /etc/nntpserver or set the
environment variable (for example in /etc/profile):</P>
<PRE>
export NNTPSERVER=localhost =
</PRE>
<HR>
<H2><A NAME="HINTS">USING LEAFNODE ON MULTI-USERS SITE.</A></H2>
<P>There are problems to avoid, especially if there are other users
on your system, or if the leafnode server is on a local network. First
you must make sure of the validity of the headers
in the outgoing posts, but you also need to limit the list of the accessible
newsgroups. Don't
forget that Fetch will download the whole content of a newsgroup if someone
tries to read it. So, be very careful with newsgroups like
<EM>alt.binaries.pictures...</EM></P>
<P>Fortunately, we can use some little awk and shell scripts to
<A HREF=#VERIF>verify and correct</A> the outgoing posts, and to
<A HREF=#LIMIT>limit the local access</A> to selected newsgroups.</P>
<H3><A NAME="VERIF">Checking the outgoing posts</A></H3>
<P>With some newsreaders, the "From:" field of the outgoing articles will
be set to <EM>myname@my.station.on.my.lan </EM> and not
<EM>myname@my.internet.provider.com</EM>.
With some of them you can configure the "From:" and "Reply-to:" fields,
while the others need you to recompile them.</P>
<P>This problem can be worse if you have a Linux box with several users.
Some of them can have misconfigured newsreader (sometimes on purpose...)
and it may be safer to check the headers of the outgoing articles before
posting them.</P>
<P>Here's a small awk filter which allows a kind of masquerading of the
"From:" line of an article. It will change the line
<CODE>"From: <EM>username@my.station.on.my.lan (user real name)</EM>"</CODE>
to a line
<CODE>"From: <EM>username@my.internet.provider.com (user real name)</EM>"</CODE>.</P>
<P>You may also ensure that <EM>username</EM> is correct (i.e. in a list
of allowed users). The same Perl script will help us to determine the
correct articles. Otherwise it will add a line "*** Wrong From field -
This article must be deleted ***" to the message.</P>
<PRE>
#! /usr/bin/gawk -f
#
# /usr/local/sbin/change_article_from_domain
#
# awk script to change the domain name on the "From:"
# line of outgoing articles.
# If the username is not valid a message will be added
# at the bottom o the file, allowing a 'grep' to delete
# him.
BEGIN {
# replace with the correct domains
local_domain="my.station.on.my.lan"
real_domain ="my.internet.provider.com"
# insert here the name of your users allowed to post articles
# (may be just one)
valid_usernames["user1"]
valid_usernames["user2"]
must_be_deleted=0
}
/^From:/ {
gsub(local_domain, real_domain)
username=substr($2,1,index ($2, "@") - 1)
if (! (username in valid_usernames)) {
must_be_deleted=1
# you can also add a system command
# example : mail to newsmaster with
# the username of the wrong article
}
}
END {
if (must_be_deleted != 0) {
print "*** Wrong From field - This article must be deleted ***"
}
}
{
print
}
</PRE>
<P>This script can be useful if you have up to, say, ten users, otherwise
you'll need to improve it in order to read the list of allowed users in
an otherfile for example.</P>
<P>Now we will execute the above script on all the outgoing articles,
sitting in /var/spool/news/out.going, then delete (or move to another
directory) those with bad usernames.</P>
<PRE>
#! /bin/bash
#
# /usr/local/sbin/modify_outgoing_articles
cd /var/spool/news/out.going
for i in * ; do
/usr/local/sbin/change_article_from_domain < $i >/tmp/modified_articles/$i
done
rm -f *
mv /tmp/modified_articles/* .
rm -f `grep -l "*** Wrong From field - This article must be deleted ***" *`
</PRE>
<P>(The backquote is used to catch the result of grep) Don't forget to create
a /tmp/modified_articles/ directory.
This script cannot prevent the fake "From:" lines, when an authorized user is
hidden behind an other
correct username. This can not be easily done, and if you really don't trust
your users, you'll have to use another Usenet package like Inn or Cnews.</P>
<P>Now all the outgoing articles will have a correct "From:" line.</P>
<H3><A NAME="LIMIT">Limiting the list of accessible news groups</A></H3>
<P>The second important point to check out is the list of fetched newsgroups.
If you haven't got a huge disk
space, it would be better to avoid downloading <EM>alt.binaries</EM> groups
or <EM>alt.2600.</EM> for example...
The problem is that fetch will download the content of each newsgroup
corresponding to a file in /var/spool/news/interesting.groups, for
example <CODE>/var/spool/news/interesting.groups/comp.os.linux.announce</CODE></P>
<P>A file in this directory is touched by leafnode every time a user tries
to read the content of the group. Are you
sure that none of your users will try to have a look at
alt.binaries.pictures.erotica...?
So there are two possible solutions:</P>
<UL>
<LI>You can edit the /usr/lib/leafnode/groupinfo file, in order to suppress
the forbidden newsgroups. But this is not a very good solution, because
fetch will re-create it during the next connection.
<LI>You can suppress the forbidden groups from
/var/spool/news/interesting.groups.
This can be done by shell script, in many different ways, for example:
</UL>
<PRE>
#! /bin/bash
#
# modify_interesting_groups
cd /var/spool/news
rm -f alt.*
rm -f *windows*
...
touch comp.os.linux.announce
touch comp.windows.x.announce
...
</PRE>
<P>you can also have a definite list of fetched newsgroups, for example in
/var/spool/news/official.list, manually created:</P>
<PRE>
# mkdir /var/spool/news/official.list
# cd /var/spool/news/official.list
# touch comp.os.linux.announce
# touch comp.os.linux.answer
# touch comp.lang.c.moderated
...
</PRE>
<P>And this list will be copied in the interesting.groups directory by a
script before each execution of fetch.</P>
<PRE>
#! /bin/bash
#
# modify_interesting_groups
cd /var/spool/news
rm -f *
copy /var/spool/news/official.list .
</PRE>
<P>Now we have two scripts to insert before running fetch. For example in
/etc/ppp/ip-up:</P>
<PRE>
#! /bin/bash
#
# /etc/ppp/ip-up
...
/usr/local/sbin/modify_interesting_groups
/usr/local/sbin/modify_outgoing_articles
fetch
</PRE>
<HR>
<H2>CONCLUSION</H2>
<P>I think that <EM>Leafnode</EM> is a very interesting package for those
(most of us) who are running
Linux on a standalone box with intermitent connection to a Usenet server,
or on a small local network
with few users. It's powerful and much simpler to install and to configure
than Inn or Cnews,
designed for bigger sites. Moreover it does not require any maintenance.</P>
<HR>
<P>Christophe Blaess (<A HREF="mailto:ccb@club-internet.fr"><EM>ccb@club-internet.fr</EM></A>).</P>
<p>
<A HREF="./index.html"><IMG SRC="../gx/indexnew.gif" ALT="[ TABLE OF CONTENTS ]"></A>
<A HREF="../index.html"><IMG SRC="../gx/homenew.gif" ALT="[ FRONT
PAGE ]"></A>
<A HREF="binstats.html"><IMG SRC="../gx/back2.gif" ALT=" Back "></A>
<A HREF="filerunner.html"><IMG SRC="../gx/fwd.gif" ALT=" Next "></A>
</BODY>
</HTML>
Reference in New Issue View Git Blame Copy Permalink