mirror of https://github.com/tLDP/LDP
344 lines
18 KiB
Plaintext
344 lines
18 KiB
Plaintext
<section id="softwarehistory" xreflabel="Usenet software: a historical perspective">
|
|
<title>Usenet software: a historical perspective</title>
|
|
|
|
<para>This section comprises excerpts from a well-known Usenet Periodic
|
|
Posting document which was last changed in Feb 1998. Our copy of that old
|
|
document was picked up from</para>
|
|
|
|
<para><literal>ftp://rtfm.mit.edu/pub/usenet-by-hierarchy/news/software/b/Usenet_Software:_History_and_Sources</literal></para>
|
|
|
|
<para>We suspect other copies will also be found elsewhere. The physical
|
|
file on the FTP server appears to have been touched last on 29 Dec
|
|
1999. The first few lines of the archived file provide information about
|
|
the origin of this document and its authors:</para>
|
|
|
|
<programlisting>
|
|
<![CDATA[
|
|
Date: Tue, 28 Dec 1999 09:00:19 GMT
|
|
Supersedes: <FMMECL.58s@tac.nyc.ny.us>
|
|
Expires: Fri, 28 Jan 2000 09:00:19 GMT
|
|
Message-ID: <FnG10J.HAo@tac.nyc.ny.us>
|
|
From: netannounce@deshaw.com (Mark Moraes)
|
|
Subject: Usenet Software: History and Sources
|
|
Newsgroups: news.admin.misc,news.announce.newusers,news.software.readers,news.software.b,news.answers
|
|
Followup-To: news.newusers.questions
|
|
Approved: netannounce@deshaw.com (Mark Moraes)
|
|
|
|
Archive-name: usenet/software/part1
|
|
Original-from: spaf@cs.purdue.edu (Gene Spafford)
|
|
Comment: edited until 5/93 by spaf@cs.purdue.edu (Gene Spafford)
|
|
Last-change: 9 Feb 1998 by netannounce@deshaw.com (Mark Moraes)
|
|
Changes-posted-to: news.admin.misc,news.misc,news.software.readers,news.software.b,news.answers
|
|
]]>
|
|
</programlisting>
|
|
|
|
<para>We have been seeing this document as a periodic posting in
|
|
<literal>news.announce.newusers</literal> since the early nineties, and
|
|
it has always been our final reference on the history of Usenet
|
|
server software. We reproduce excerpts below, retaining the portions
|
|
which discuss server software, and removing discussions of client
|
|
software, newsreaders, software for non-Unix operating systems,
|
|
<emphasis>etc.</emphasis> All quoted portions are reproduced unedited
|
|
other than changing FTP file paths to the modern URL format. We have
|
|
added our comments emphasised, in separate paragraphs. We feel the
|
|
information captured here is essential reading for anyone interested in
|
|
Usenet server software.
|
|
</para>
|
|
|
|
<para>If anyone can point us to a fresher version of this document, in
|
|
case it is still maintained, we will be happy to refer to that version
|
|
instead of this one, though we suspect the reader will not suffer due to
|
|
the four-year gap; most of the information reproduced below is
|
|
historical anyway.
|
|
</para>
|
|
|
|
<section><title>The quoted excerpts</title>
|
|
|
|
<para>Currently, Usenet readers interact with the news using a number of
|
|
software packages and programs. This article mentions the important
|
|
ones and a little of their history, gives pointers where you can look
|
|
for more information and ends with some special notes about ``foreign''
|
|
and ``obsolete'' software. At the very end is a list of sites from which
|
|
current versions of the Usenet software may be obtained.
|
|
</para>
|
|
|
|
<para> ... </para>
|
|
|
|
|
|
<section><title>History</title>
|
|
|
|
<para>Usenet came into being in late 1979, shortly after the release of V7
|
|
Unix with UUCP. Two Duke University grad students in North Carolina,
|
|
Tom Truscott and Jim Ellis, thought of hooking computers together to
|
|
exchange information with the Unix community. Steve Bellovin, a grad
|
|
student at the University of North Carolina, put together the first
|
|
version of the news software using shell scripts and installed it on the
|
|
first two sites: <literal>unc</literal> and <literal>duke</literal>. At
|
|
the beginning of 1980 the network consisted of those two sites and
|
|
<literal>phs</literal> (another machine at Duke), and was described
|
|
at the January Usenix conference. Steve Bellovin later rewrote
|
|
the scripts into C programs, but they were never released beyond
|
|
<literal>unc</literal> and <literal>duke</literal>. Shortly thereafter,
|
|
Steve Daniel did another implementation in C for public distribution.
|
|
Tom Truscott made further modifications, and this became the ``A'' news
|
|
release.</para>
|
|
|
|
<para>In 1981 at U. C. Berkeley, grad student Mark Horton and high school
|
|
student Matt Glickman rewrote the news software to add functionality
|
|
and to cope with the ever increasing volume of news -- ``A'' News was
|
|
intended for only a few articles per group per day. This rewrite was
|
|
the ``B'' News version. The first public release was version 2.1 in
|
|
1982; the 1.* versions were all beta test. As the net grew, the news
|
|
software was expanded and modified. The last version maintained and
|
|
released primarily by Mark was 2.10.1.</para>
|
|
|
|
<para>Rick Adams, at the Center for Seismic Studies, took over
|
|
coordination of the maintenance and enhancement of the B News software
|
|
with the 2.10.2 release in 1984. By this time, the increasing volume
|
|
of news was becoming a concern, and the mechanism for moderated groups
|
|
was added to the software at 2.10.2. Moderated groups were inspired by
|
|
ARPA mailing lists and experience with other bulletin board systems.
|
|
In late 1986, version 2.11 of B News was released, including a number
|
|
of changes to support a new naming structure for newsgroups, enhanced
|
|
batching and compression, enhanced <literal>ihave/sendme</literal>
|
|
control messages, and other features.</para>
|
|
|
|
<para>The final release of B News was 2.11, patchlevel 19. B News has
|
|
been declared ``dead'' by a number of people, including Rick Adams, and
|
|
is unlikely to be upgraded further; most Usenet sites are using C News
|
|
or INN (see next paragraphs).</para>
|
|
|
|
<para>In March 1986 a package was released implementing news transmission,
|
|
posting, and reading using the Network News Transfer Protocol (NNTP)
|
|
(as specified in RFC 977). This protocol allows hosts to exchange
|
|
articles via TCP/IP connections rather than using the traditional UUCP.
|
|
It also permits users to read and post news (using a modified news user
|
|
agent) from machines which cannot or choose not to install the Usenet
|
|
news software. Reading and posting are done using TCP/IP messages to a
|
|
server host which does run the Usenet software. Sites which have many
|
|
workstations like the Sun and SGI, and HP products find this a convenient
|
|
way to allow workstation users to read news without having to store
|
|
articles on each system. Many of the Usenet hosts that are also on the
|
|
Internet exchange news articles using NNTP because the load impact of NNTP
|
|
is much lower than UUCP (and NNTP ensures much faster propagation).</para>
|
|
|
|
<para><emphasis>Our comments: This remark about relative loadings
|
|
of UUCP and NNTP is no longer applicable with faster machines and
|
|
networks, and with hugely increased traffic volumes. Today's desktop
|
|
computers, let alone servers, can all handle both NNTP and UUCP loads
|
|
effortlessly, if traffic volumes can be restricted. This is partly
|
|
due to performance enhancements to UUCP as embodied in Taylor UUCP,
|
|
and partly due to vastly faster processors.</emphasis></para>
|
|
|
|
<para>NNTP grew out of independent work in 1984-1985 by Brian Kantor
|
|
at U. C. San Diego and Phil Lapsley at U. C. Berkeley. Primary
|
|
development was done at U. C. Berkeley by Phil Lapsley with help from
|
|
Erik Fair, Steven Grady, and Mike Meyer, among others. The NNTP package
|
|
(now called the reference implementation) was distributed on the 4.3BSD
|
|
release tape (although that was version 1.2a and out-of-date) and is
|
|
also available on many major hosts by anonymous FTP. The current
|
|
version is 1.5.12.2. It includes NOV (News Overview -- see below)
|
|
support and runs on a wide variety of systems. It is available from
|
|
<literal>ftp.academ.com:/pub/nntp1.5/nntp.1.5.12.2.tar.gz</literal>.
|
|
For those with access to the World-Wide Web on the Internet, the
|
|
WWW page <literal>http://www.academ.com/academ/nntp.html</literal>
|
|
contains a description and news about NNTP. A different
|
|
variant, called nntp-t5, implements many of the extensions
|
|
provided by INN (including NOV support). It is available from
|
|
<literal>ftp.uu.net:/networking/news/nntp/nntp-t5.tar.gz</literal>.
|
|
</para>
|
|
|
|
<para>One widely-used version of news, known as C News, was developed
|
|
at the University of Toronto by Geoff Collyer and Henry Spencer. This
|
|
version is a rewrite of the lowest levels of news to increase article
|
|
processing speed, decrease article expiration processing and improve the
|
|
reliability of the news system through better locking, etc. The package
|
|
was released to the net in the autumn of 1987. For more information,
|
|
see the paper ``News Need Not Be Slow,'' published in The Winter 1987
|
|
Usenix Technical Conference proceedings. This paper is also available
|
|
from <literal>ftp://ftp.cs.toronto.edu/doc/programming/c-news.*,</literal>
|
|
and is recommended reading for all news software programmers. The most
|
|
recent version of C News is the Sept 1994 ``Cleanup Release.'' C News
|
|
can be obtained by anonymous ftp from its official archive site,
|
|
<literal>ftp.cs.toronto.edu:pub/c-news/c-news.tar.Z</literal>. </para>
|
|
|
|
<para><emphasis>Our comments: C News is no longer maintained by
|
|
anyone that we know, other than ourselves. However, after fixing
|
|
the remaining bugs in the source, we have not found the need for
|
|
further maintenance. NNTPd from Brian Kantor and Phil Lapsley is
|
|
in the same state, but we are working on enhancements to the source
|
|
for access control and other functionality.</emphasis></para>
|
|
|
|
<para>Another Usenet system, known as InterNetNews, or INN, was written by
|
|
Rich Salz <literal>(rsalz@uunet.uu.net)</literal>. INN is designed to run
|
|
on Unix hosts that have a socket interface. It is optimized for larger
|
|
hosts where most traffic uses NNTP, but it does provide full UUCP support.
|
|
INN is very fast, and since it integrates NNTP many people find it easier
|
|
to administer only one package. The package was publicly released on
|
|
August 20, 1992. For more information, see the paper ``InterNetNews:
|
|
Usenet Transport for Internet Sites'' published in the June 1992
|
|
Usenix Technical Conference Proceedings. INN can be obtained
|
|
from many places, including the 4.4BSD tape; its official
|
|
archive site is <literal>ftp.uu.net</literal> in the directory
|
|
<literal>/networking/news/nntp/inn</literal>. Rich's last official
|
|
release was 1.4sec in Dec 1993.</para>
|
|
|
|
<para><emphasis>Our comments: The original paper by Rich Salz about
|
|
INN, where he proposed the design of an alternate Usenet server
|
|
software, is a must-read for readers interested in Usenet server
|
|
software. So is the paper by C News authors, cited before it. Most of
|
|
the issues that Rich Salz had with C News, as stated in his paper,
|
|
were very relevant at that time. Today, with the current version of
|
|
NNTPd and the incorporation of the message ID daemon and NOV, these
|
|
issues are no longer relevant, and the choice of C News+NNTPd versus
|
|
INN is now based more on the level of maintenance of source code,
|
|
familiarity and personal preferences than on core design factors.
|
|
</emphasis></para>
|
|
|
|
<para>In June 1995, David Barr began a series of unoffical releases
|
|
of INN based on 1.4sec, integrating various bug-fixes, enhancements
|
|
and security patches. His last release was 1.4unoff4, found in
|
|
<literal>ftp://ftp.math.psu.edu:/pub/INN</literal>. This site is also the
|
|
home of contributed software for INN and other news administration
|
|
tools.</para>
|
|
|
|
<para>INN is now maintained by the Internet Software Consortium
|
|
<literal>(inn@isc.org)</literal>. The official INN home is now
|
|
<literal>http://www.isc.org/isc/</literal> and the latest version (1.7.2)
|
|
can be obtained from <literal>ftp://ftp.isc.org/isc/inn/</literal>.</para>
|
|
|
|
<para><emphasis>Our comments: The URL for the INN home page above
|
|
is probably incorrect. Try http://www.isc.org/products/INN/.
|
|
</emphasis></para>
|
|
|
|
<para>Towards the end of 1992, Geoff Collyer implemented NOV (News
|
|
Overview): a database that stores the important headers of all news
|
|
articles as they arrive. This is intended for use by the implementors
|
|
of news readers to provide fast article presentation by sorting and
|
|
``threading'' the article headers. (Before NOV, newsreaders like
|
|
<literal>trn</literal>, <literal>tin</literal> and <literal>nn</literal>
|
|
came with their own daemons and databases that used a nontrivial amount
|
|
of system resources). NOV is fully supported by C News, INN and NNTP-t5.
|
|
Most modern news readers use NOV to get information for their threading
|
|
and article menu presentation; use of NOV by a newsreader is fairly easy,
|
|
since NOV comes with sample client-side threading code.
|
|
</para>
|
|
|
|
<para> ... </para>
|
|
|
|
<para>Details on many other mail and news readers for MSDOS, Windows and
|
|
OS/2 systems can be found in the FAQ posted to
|
|
<literal>comp.os.msdos.mail-news</literal>.</para>
|
|
|
|
<programlisting>
|
|
<![CDATA[
|
|
<ftp://rtfm.mit.edu/pub/usenet/comp.os.msdos.mail-news/intro>
|
|
<ftp://rtfm.mit.edu/pub/usenet/comp.os.msdos.mail-news/software>
|
|
]]>
|
|
</programlisting>
|
|
|
|
</section>
|
|
|
|
<section><title>Newsfeed management software</title>
|
|
|
|
<para><emphasis role=bold>Gup</emphasis>, the Group Update Program is a Unix
|
|
mail-server program that lets a remote site change their newsgroups
|
|
subscription on their news feed without requiring the intervention of
|
|
the news administrator at the feed site. Gup operates with the INN
|
|
(and likely the C News) batching mechanisms. The news administrators
|
|
at the remote sites simply mail commands to gup to make changes to
|
|
their own site's subscription list. The mail/interface is password
|
|
protected. Gup checks the requests for valid newsgroup names,
|
|
patterns that have no effect and so on. Gup's authors are Mark
|
|
Delany (<literal>markd@mira.net.au</literal>) and Andrew Herbert
|
|
(<literal>andrew@mira.net.au</literal>). Its official FTP location
|
|
is <literal>ftp.mira.net.au:/unix/news/gup-0.4.tar.gz</literal>,
|
|
but since that's not as well connected as UUNET, people are strongly
|
|
advised to obtain it from a mirror site, <emphasis>e.g.</emphasis>
|
|
<literal>ftp.uu.net:/networking/news/misc/gup-0.4.tar.gz</literal>.</para>
|
|
|
|
<para><emphasis role=bold><literal>dynafeed</literal></emphasis> is
|
|
a package from Looking Glass Software Limited that maintains a
|
|
<literal>.newsrc</literal> for every remote site and generates the batches
|
|
for them. Remote sites can use UUCP or run a program to change their
|
|
<literal>.newsrc</literal> dynamically. It comes with a program that the
|
|
remote site can run to monitor readership in newsgroups and dynamically
|
|
update the feed list to match reader interest. The goal of this is
|
|
to get a feed that sends only exactly the groups currently being read.
|
|
<literal>dynafeed</literal> can be obtained from
|
|
<literal>ftp://ftp.clarinet.com/sources/dynafeed.tar.Z</literal>.</para>
|
|
|
|
</section>
|
|
|
|
<section><title>News processing software</title>
|
|
|
|
<para>Software also exists to automatically archive Usenet newsgroups.
|
|
The package <literal>rkive</literal>, written by Kent Landfield
|
|
(<literal>kent@sterling.com</literal>) can be configured to archive
|
|
news automatically based on different headers -- Archive-Name,
|
|
Volume-Issue, Chronological, Subject and External-Command to
|
|
name a few. It can be run in batch mode from the command line or
|
|
from cron. It can also be installed in the <literal>sys</literal>
|
|
or <literal>newsfeeds</literal> file to process articles as they are
|
|
received. <literal>rkive</literal> supports local spool directories as
|
|
well as NNTP based access. <literal>rkive</literal> is available via
|
|
FTP from <literal>ftp://ftp.sterling.com/rkive</literal>. </para>
|
|
|
|
<para>Newsclip is a programming language for writing news
|
|
filtering programs, from Looking Glass Software Limited, marketed
|
|
by ClariNet Communications Corp. It is C-like, and translates to
|
|
C, so a C compiler is required. It has data-types to represent
|
|
the kinds of things found in article headers and bodies. It can
|
|
maintain databases of users, message-ids, patterns, subjects, etc.
|
|
These can be used to decide whether to ignore or select an article.
|
|
Newsclip can either operate as a standalone program or as part
|
|
of rn. It is free for non-commercial use and is available from
|
|
<literal>ftp://ftp.clarinet.com/sources/nc.tar.Z</literal>. Contact
|
|
<literal>clari-info@clarinet.com</literal> with a subject line of
|
|
``newsclip'' for more info.
|
|
</para>
|
|
|
|
</section>
|
|
|
|
<section><title>Commercial software</title>
|
|
|
|
<para>DNEWS is a commercial product from NetWin. DNEWS licenses
|
|
are provided free to educational institutions for non profit
|
|
use. With DNEWS, the news is stored in a database so as not to
|
|
overload the raw file system. DNEWS supports 'sucking' where only
|
|
groups which users read are pulled over from the feeder site. DNEWS
|
|
is currently known to run on VMS, Windows NT, Solaris, SunOS,
|
|
Unixware, HP/UX. DNEWS binaries are available by anonymous FTP from
|
|
<literal>ftp://ftp.std.com/ftp/vendors/netwin/dnews</literal> or from
|
|
<literal>http://world.std.com/~netwin/</literal> DNEWS sources can be
|
|
obtained on request, see the file <literal>source.txt</literal> in the
|
|
FTP area for more information.</para>
|
|
|
|
<para><emphasis>Our comments: The information on DNEWS may be dated. We
|
|
have been seeing DNEWS on their own Website for quite a few years now.
|
|
Check <literal>www.netwinsite.com</literal>. Moreover, there are other
|
|
commercial Usenet server software systems available, including the one
|
|
bundled with the Internet Information Server of Microsoft Windows NT and
|
|
the ones from iPlanet. And for carrier class systems, there are many
|
|
commercial Usenet routers available.
|
|
</emphasis></para>
|
|
|
|
</section>
|
|
|
|
<section><title>Special note on ``notes'' and old versions of news</title>
|
|
|
|
<para> ... </para>
|
|
|
|
<para>``B'' news software is currently considered obsolete. Unix sites
|
|
joining the Usenet should install C news or INN to ensure proper
|
|
behavior and good performance. Most old B news software had
|
|
compiled-in limits on the number of newsgroups and the number of
|
|
articles per newsgroup; the increasing volume of news means that B
|
|
news software cannot reliably cope with a moderately-full newsfeed.
|
|
</para>
|
|
</section>
|
|
|
|
</section>
|
|
|
|
</section>
|