239 lines
9.7 KiB
HTML
239 lines
9.7 KiB
HTML
<!doctype html public "-//w3c//dtd html 3.2 Final//EN">
|
|
|
|
<html>
|
|
|
|
<head>
|
|
<title>Automated News Fetching</title>
|
|
</head>
|
|
|
|
<body bgcolor="#ffefd5" text="#181871">
|
|
|
|
<H4>
|
|
"Linux Gazette...<I>making Linux just a little more fun!</I>"
|
|
</H4>
|
|
|
|
<P> <HR> <P>
|
|
<!--===================================================================-->
|
|
<center>
|
|
<H1>Slrn and Slrnpull: Sucking Down The News</H1>
|
|
</center>
|
|
|
|
<center><h4><a href="mailto: layers@vax2.rainis.net">by Larry
|
|
Ayers</a></h4></center>
|
|
|
|
<P> <HR>
|
|
<!--===================================================================-->
|
|
<p>
|
|
There are quite a few methods of reading Usenet postings. A conventional
|
|
newsreader will log on to your remote server, download headers of the new
|
|
messages in groups you want to follow, then allow you to tag the messages you
|
|
want to read. These messages are then fetched for you. All of this happens
|
|
while online, and the time can mount up.</p>
|
|
|
|
<p>
|
|
Another approach is one used by Suck and Leafnode, among others. These
|
|
programs are designed to be used non-interactively and usually are set up to
|
|
deposit fetched postings into a local spool-directory. Suck requires that you
|
|
have an active news-server, such as INN or CNEWS, on your machine. Leafnode
|
|
doesn't need the news-server (it has its own), but both programs are designed
|
|
for multiple users and might be overkill for single-user machines.</p>
|
|
|
|
<p>
|
|
Slrn is a popular text-mode newsreader, written by <a href="mailto:
|
|
davis@space.mit.edu">John Davis</a> at MIT. It originally belonged to the
|
|
first category above, but recently Davis has been working on an extension for
|
|
Slrn which will pull down messages from a server and store them locally. The
|
|
messages can then be read offline with Slrn. The extension is called
|
|
Slrnpull, and it comes with the most recent beta version of Slrn.</p>
|
|
|
|
<center><h3>Installation and Usage</h3></center>
|
|
|
|
<p> If you have the S-lang library on your system, you can compile Slrn and
|
|
Slrnpull from the source, which is available (along with the S-lang library
|
|
source) from <a href="ftp://space.mit.edu /pub/davis/slrn">this site.</a>
|
|
A binary, statically-linked version may be in the /pub/Linux/Incoming
|
|
directory at sunsite.unc.edu by the time you read this. If you prefer a
|
|
certain location for the news-spool directory (which can get large) the
|
|
<i>slrnfeat.h</i> file in the /slrn/src directory can be edited.</p>
|
|
|
|
<p>
|
|
Slrn uses a <em>configure</em> script which should enable it to be compiled on
|
|
most Linux systems. Once you've put the executables in a directory on your
|
|
path, create the spool directory (<i>/var/spool/news/slrnpull</i> or whatever
|
|
you've defined it to be), then copy the supplied sample script
|
|
<i>slrnpull.conf </i> to the new directory. This needs to be edited before
|
|
you start Slrnpull for the first time. The format is not complicated; here are
|
|
John Davis' comments from the sample file:<br><code><pre>
|
|
|
|
# The syntax of the file is very simple.
|
|
# Any line that is blank or begins with a '#' character will be ignored by
|
|
# slrnpull. The remaining lines consist of 1-3 fields separated by
|
|
# whitespace:
|
|
#
|
|
# NEWSGROUP_NAME MAX_ARTICLES_TO_RETRIEVE NUMBER_OF_DAYS_BEFORE_EXPIRE
|
|
#
|
|
# The first field must contain the name of a newsgroup.
|
|
#
|
|
# The second field denotes the number of articles to retrieve for the
|
|
# newsgroup; if its value is 0, all available articles will
|
|
# be retrieved.
|
|
#
|
|
# The third field indicates the number of days after an article is retrieved
|
|
# before it will be eligible for deletion. If this value is 0, articles from
|
|
# this group will not expire.
|
|
#
|
|
#
|
|
# If a field is blank, or contains the single character '*', default values
|
|
# will apply to the field. Defaults may be set by a line whose newsgroup
|
|
# field is 'default'. Such a line will denote default values to be applied to
|
|
# the lines following it or until another default is established.
|
|
|
|
# For example:
|
|
default 20 14
|
|
# indicates a default value of 20 articles to be retrieved from the server and
|
|
# that such an article will expire after 14 days.
|
|
comp.os.linux.misc 50 7
|
|
comp.os.linux.x 20 7
|
|
comp.os.linux.announce * *
|
|
</pre>
|
|
</code></p>
|
|
|
|
<p>
|
|
This is easier to set up than some news programs I've used!</p>
|
|
|
|
<p>
|
|
Assuming you have the $NNTPSERVER variable set to your news-server's IP
|
|
address in your <i>~/.bash_profile</i> or <i>~/.cshenv</i> file, Slrnpull
|
|
should be ready to try out. The first time you start it up it will create a
|
|
subdirectory for each news-group you have specified. Then it will log on to
|
|
your server and download messages, displaying the connection speed and number
|
|
of articles on your terminal screen.</p>
|
|
|
|
<p>
|
|
You probably subscribe to certain groups for which you want all of the new
|
|
messages. For certain others you may want to be more selective in what you
|
|
download. A kill-file can be created in the spool directory which specifies,
|
|
on a per-group basis, which messages you would prefer be left on the
|
|
server.</p>
|
|
|
|
<p>
|
|
Starting up Slrn with the switch <i>--spool</i> will cause it to load the
|
|
contents of your newly-filled spool-file. Reading messages this way is fast,
|
|
and any which you delete will then be invisible in the newsreader, though they
|
|
remain on the disk until they are expired. Any follow-up postings which you
|
|
might write are stored in a subdirectory of the spool. The next time you run
|
|
Slrnpull it will upload them to the server before retrieving new messages.</p>
|
|
|
|
<p>
|
|
Slrnpull keeps a log of all transactions to the server; these messages are
|
|
displayed on the screen as the program runs, but the idea of this program is
|
|
that you don't need to be sitting there watching. The log is useful for
|
|
checking to see if your postings have been accepted by the server.</p>
|
|
|
|
<p>
|
|
Periodically Slrnpull should be run with the <i>--expire</i> switch, which
|
|
will remove all messages you've marked for deletion while reading news with
|
|
Slrn. This could be run every night as a cron job.</p>
|
|
|
|
<p>
|
|
It will take some fine-tuning of the <i>slrnpull.conf</i> file, but eventually
|
|
you will have the program retrieving just the messages you want. It might seem
|
|
like a waste to be downloading all of the junk messages along with the
|
|
worthwhile ones, but it's a continuous process and doesn't take long. I've
|
|
found that running Slrnpull while browsing the web or receiving an FTP file
|
|
works well.</p>
|
|
|
|
|
|
<p>The sample <i>.slrnrc</i> file included with the program has an if/then
|
|
statement which causes Slrn to read the local active file when run in spool
|
|
mode, while keeping Slrn in standard mode from retrieving the bulky remote
|
|
active file each time a connection is made. This lets you read news
|
|
directly from your server when desired.
|
|
|
|
<p> The sample file includes some new entries in order for Slrn to make use
|
|
of the spooled messages. These are:<br></p>
|
|
|
|
<pre>
|
|
<code></code>
|
|
|
|
set spool_inn_root "/var/spool/news/slrnpull"
|
|
set spool_root "/var/spool/news/slrnpull/news"
|
|
set spool_nov_root "/var/spool/news/slrnpull"
|
|
set use_slrnpull 1
|
|
hostname "your.host.name"
|
|
username "your_user_name"
|
|
|
|
</pre>
|
|
|
|
<p>The remainder of the <i>.slrnrc</i> file is the same as in previous Slrn
|
|
versions, so if you already have one customized to your liking the
|
|
Slrnpull-specific sections can be lifted from the sample and pasted in.
|
|
|
|
<p>
|
|
I initially had some trouble convincing slrnpull to talk to my news-server. I
|
|
asked John Davis for help and he sent me a patch for one source file which
|
|
caused slrnpull to generate a debugging log; from the logfile he determined
|
|
that the problem was with the proprietary Dnews server software which my
|
|
provider uses. The currently available version has this patch included.</p>
|
|
<p>
|
|
|
|
If you want to find out what software your news-server uses, just telnet into
|
|
the news machine:<br></p>
|
|
<pre>
|
|
|
|
<i>telnet [IP address] :nntp</i>
|
|
|
|
</pre>
|
|
|
|
<p>
|
|
The server will identify itself when you log in.</p>
|
|
|
|
<center><h3>Conclusion</h3></center>
|
|
|
|
<p>
|
|
Slrnpull is probably most useful with low-volume newsgroups, such as
|
|
<em>comp.os.linux.announce</em>. You would most likely want to see all of the
|
|
messages anyway in such a group and Slrnpull will fetch them all. High-volume
|
|
groups, such as <em>comp.os.linux.advocacy</em>, typically have a high
|
|
chaff-to-wheat ratio, and in these a quick scan of the headers for the few of
|
|
interest (while online) might be more efficient. Slrnpull is also effective
|
|
for obtaining a quick idea of the the flavor and tone of a group: just tell
|
|
it to suck down the most recent twenty messages in the group, and see
|
|
what you think. </p>
|
|
|
|
<p>If you have never used Slrn, I highly recommend this program, especially
|
|
if you read news over a PPP or SLIP connection. It's fast and efficient,
|
|
and its behaviour can be easily molded to your needs. Users of the Emacs
|
|
news interface Gnus will find the transition painless, as most of the
|
|
keystroke commands are identical. Gnus has many more features but it's
|
|
slower to use over a network and is much more demanding of system resources.
|
|
</p>
|
|
|
|
<hr>
|
|
|
|
<address>
|
|
<a href="http://vax2.rainis.net/~layers/">Larry
|
|
Ayers<layers@vax2.rainis.net></a></address>
|
|
<p>
|
|
<!-- hhmts start -->
|
|
Last modified: Thu Feb 27 18:39:52 CST 1997
|
|
<!-- hhmts end -->
|
|
|
|
<!--===================================================================-->
|
|
<P> <hr> <P>
|
|
<center><H5>Copyright © 1997, Larry Ayers <BR>
|
|
Published in Issue 15 of the Linux Gazette, March 1997</H5></center>
|
|
|
|
<!--===================================================================-->
|
|
<P> <hr> <P>
|
|
<A HREF="./index.html"><IMG ALIGN=BOTTOM SRC="../gx/indexnew.gif"
|
|
ALT="[ TABLE OF CONTENTS ]"></A>
|
|
<A HREF="../index.html"><IMG ALIGN=BOTTOM SRC="../gx/homenew.gif"
|
|
ALT="[ FRONT PAGE ]"></A>
|
|
<A HREF="./amaya.html"><IMG SRC="../gx/back2.gif"
|
|
ALT=" Back "></A>
|
|
<A HREF="./sigrot.html"><IMG SRC="../gx/fwd.gif" ALT=" Next "></A>
|
|
<P> <hr> <P>
|
|
</BODY>
|
|
</HTML>
|